Perl Email Project: Not Dead
This list has been quiet for quite some time, as has the Perl Email Project. On one hand, that's given the project a lot of time to have coders shake it around and see what's what. On the other, that means that a lot of people may have the sneaking suspicion that PEP has reached that form of stability known as death. Fear not! PEP is not dead, it was only resting. A few email-obsessed members of the Perl community, including PEP's last caretaker, Casey West, met at YAPC::NA::2006 to talk about the project's past, present, and future. There was a lot of talk about how we can not only revive, but also improve the project. First and foremost, we want to let people know that it's not dead. One way to do that is by yelling, "Hey! It's not dead!" Well, pretend I'm yelling this message -- the mixed case is just a kindness for your eyes. A better way to do this is by actually updating our code. This morning I released Email::Send 2.10, fixing a number of bugs in the RT queue. I'm hoping to get all the backlog of critical bugs closed in the next two weeks -- really, I think that getting it done this week is realistic, for most of the bugs. We'd also like to start expanding the scope of the project to include not just "write new, simple Email:: modules," but also things like: * build a knowledge base for solving email problems in Perl * fix critical bugs in old (and possibly superseded) but popular code * build up test data that can be used by all email module authors In other words, we want to make Perl an even better language for email handling than it already is, by providing new tools, maintaining old tools, and sharing our wisdom. All of PEP's code will soon be in one central repository, and we will make it easy to join the project as a coder. In the meantime, we've created a PEP wiki, where we will be publishing plans, known problems and solutions, and general information about the project. You can currently find the wiki at http://pep.pobox.com/wiki although we'd like to move it to a perl.org domain soon. Please consider this a promise, at the very least from me, that PEP is going to be getting a lot of much-needed care. If you know of specific things that need attention, file a bug, or send an email to the list (or to me). -- rjbs signature.asc Description: Digital signature
who loves writing tests?
Is it you? I hope so! Email:: needs more tests. Here's coverage on the worst offenders: Email-Address : 74.4% Email-Abstract: 67.9% Email-Filter : 64.7% Email-Send: 60.5% Email-LocalDelivery: 51.9% Today, an important bug in Email::Send was reported that snuck by because of insufficient testing. One major feature, message_modifier, was completely untested. More takes makes it easy to more reliably fix bugs and improve interoperability. If anyone reading this mail thinks, "Wow! Writing tests for the Email::* modules would be totally awesome!" then please do so! Over the next week or two (or three) I'll be working on some new tools for testing these modules, but in the meantime, simple tests of "does this feature exist and work in the most likely case" would be fantastic. -- rjbs pgp9wIZ3BD1QU.pgp Description: PGP signature
PEP IRC BBQ
From now on, I will be findable in #email on irc.perl.org. I might not be at my console, but I'll be in the channel. Show up if you're afraid of sending email. (If you're afraid of sending email... maybe PEP can help!) -- rjbs signature.asc Description: Digital signature
Email::Send::Sendmail availability
So, here's the first of many ponderings to come about Email::Send's interface... What's the deal with Email::Send::Sendmail being "always available." There is a test to ensure that it is, which makes it weirder than just a bug. Right now, the code says, "If you can find the sendmail program, the mailer is available. If you cannot, the mailer is available, but the success message should look like a failure message." WTF? Why shouldn't the Sendmail mailer just say it's not available? -- rjbs signature.asc Description: Digital signature
Re: PEP IRC BBQ
* "Karen J. Cravens" <[EMAIL PROTECTED]> [2006-07-06T14:52:23] > I deleted the message where you mentioned it, but... is > Email::LocalDelivery really still a going concern? Shouldn't it be (or > isn't it already) rolled into Email::Send? Or am I completely > misremembering (I stopped using Email::LD when I switched to SQL-based > storage)? They're different. Email::Send takes a message and fires it out into the internet like a circus performer from a cannon. Email::LocalDelivery drops a message into a file on your local disk like ice cream from a sad child's cone. In other words: Email::Send is like sendmail(1) and Email::LocalDelivery is like deliver(8). Any program that wants to write a message to an mbox or Maildir wants E::LD. Any program that wants to send mail to another host wants Email::Send. I am interested in writing a LocalDelivery mailer for Email::Send, however, that will send mail to local storage. Useful! -- rjbs signature.asc Description: Digital signature
Email-Send 2.15 in Subversion
http://pep.pobox.com/svn/Email-Send/trunk/
It's not headed to CPAN just yet. There are a few very minor changes I've made
to its behavior, and I want to do a bit more testing.
The most major is that Email::Send::IO now appends, rather than prints, to its
target. If it's using STDERR or STDOUT, which are the default and "documented
alternative" this is not actually a change. If delivering to a file, however,
this means that if you deliver many messages to a file, they'll all be there.
Previously it would clobber the file's content on each new delivery.
Everything else should be minor changes:
- added simple test for message modifier (response to bug from ABH)
- use File::Spec->path for path, not ENV{PATH} (bug 20109, Simon Flack)
- append, to not print, to IO::All objects
- use Symbol.pm, not global filehandles
- plan all tests
- remove use warnings
- undef is never a valid message
- improve testing
With this release, all but one (wishlist) bug is closed and Email/Send.pm is at
100% test coverage! Too bad that doesn't mean all its bugs are fixed.
I'll be testing with this jsut a bit longer, then releasing it this week.
After that, I'll post a more detailed roadmap, including a schedule for some
possibly non-back-compat changes in the future. (The breakage should very
minor! Don't panic!)
--
rjbs
pgpiu6CdgMgPM.pgp
Description: PGP signature
Re: PEP IRC BBQ
* "Karen J. Cravens" <[EMAIL PROTECTED]> [2006-07-06T14:52:23] > I deleted the message where you mentioned it, but... is > Email::LocalDelivery really still a going concern? Shouldn't it be (or > isn't it already) rolled into Email::Send? Or am I completely > misremembering (I stopped using Email::LD when I switched to SQL-based > storage)? They're different. Email::Send takes a message and fires it out into the internet like a circus performer from a cannon. Email::LocalDelivery drops a message into a file on your local disk like ice cream from a sad child's cone. In other words: Email::Send is like sendmail(1) and Email::LocalDelivery is like deliver(8). Any program that wants to write a message to an mbox or Maildir wants E::LD. Any program that wants to send mail to another host wants Email::Send. I am interested in writing a LocalDelivery mailer for Email::Send, however, that will send mail to local storage. Useful! -- rjbs signature.asc Description: Digital signature
Email::MIME modifier/creator bugs... fixed?
I've been working my way through Email::'s RT queues, and by far the most perplexing bugs (so far) were on Email::MIME::Modifier. One, which showed up elsewhere, regarded the fact that things would fall over if you tried to set an empty set of parts. I think that's fixed, and I think it was straightforward, in the end. The other had to do with the automatic encoding of binary parts into base64. Basically, single-part messages wouldn't pick up the content-transfer-encoding of their only part, they'd just drop it. That meant that if you tried to send a one-part message containing, say, a gzip file, it would be naked 8-bit. Oops! Fixing this revealed that all parts, binary or otherwise, were set to be base64 encoded. I've fixed this by only setting encoding to base64 for binary attachments (detected by looking for NUL or 8-bit characters) and by keeping the content-transfer-encoding of the single part when parts_set-ing to one part. I've written tests and examined output, but this change is worth having some more eyes on. Still, I think everything will be Just Fine. -- rjbs signature.asc Description: Digital signature
Re: using a "PreferredMailer", unsure about mailer_args.
* Simon Flack <[EMAIL PROTECTED]> [2006-07-07T10:01:39]
> 1. Email::Policy::preferred_mailer() *
> 2. $Email::Send::PreferredMailer::Mailer
> 3. $ENV{EMAIL_SEND_PREFERRED_MAILER}
We have a similar system, actually, but we already implemented "standard mailer
args" -- for some value of "standard." I've attached a slightly bowdlerized
version of the module.
It has the benefit of letting you do something like this:
sendmail $message => { env_from => $env_from, mailer => 'SMTP' };
...but if the environment variable was set, it ignores the mailer, there. This
means that you can run a program like this:
$ SENDMAIL_MAILER=SQLite SENDMAIL_MAILER_db_name=emails.db some-command
...and have /all/ emails delivered to the database, rather than to customers.
> We're now running into a problem that the mailer @args differ from
> mailer to mailer. This is also potentially a problem the built-in (3)
> which tries all mailers.
Ugh, tell me about it!
I submitted a number of patches to Email::Send this past winter, and now that
I've taken over maintenance, some of those have been built in. The two that
matter, here, are the ability to use mailers that don't start with
"Email::Send::" (which the documentation previously claimed was possible, but
was not), and the ability to use objects as mailers.
This let us create a hierarchy of mailers that DO have a common set of args.
Another /highly important/ patch that I have /not/ yet applied can be seen
here: http://rt.cpan.org/Ticket/Display.html?id=19733
It lets you do this:
$sender->send($message, { ...mailer_args...}, {...modifier_args... });
Currently, one can't send arguments to the mailer on a per-message basis, which
means that setting things like env_to are not possible.
In my opinion, standardizing on a core set of arguments is not reasonable until
this is fixed, as too many of them will turn out to be things we want to send
per-message.
It looks like you agree with me, for the most part. What do you think of the
above, specifically?
> args, at the cost of breaking backwards compatability & adding a little
> complexity to the mailers themselves? I'd be happy to put
> option-marshalling in PreferredMailer, but that would limit it to
> specific mailers, and would need to be updated for each new mailer.
Oh, backcompat, how you bite at my ankles! I fear that Email::Send may have
too many small design flaws that make the number of backcompat problems awfully
obnoxious.
1. inconsistent args to mailers
2. Too many things passed as lists, not refs (modifier args, for example).
These make it hard to add new semantics, as above. After all, if your
current message modifier wants \%, the above patch will break it.
3. there is no space to grow in the namespace. Email::Send::* is assumed to
be a mailer.
I wonder if the best thing isn't to figure out all the big problems and then
decide whether the total backcompat breakage is acceptable. If not, we can
decide which few to allow in, and we can also decide whether we need, say,
Email::Sender.
FWIW, we've long ago forked Email::Send internally, screwing backwards compat,
to add that patch and some others. We have an internal, common interface for
all our mailers, using Exception::Class instead of Return::Value to return
results. I'm OK with sticking with Return::Value (and adding a ->die method,
probably), but what we got from E::C is the requirement to use one set of
classes. Did your message fail to get out at all?
Email::SendX::Exception::Failure. Did it get out to some?
Email::SendX::Exception::Sucess, with fields for failed recipients.
The benefits of the guaranteed pluggability have been great.
> * Optional. Email::Policy is similar to File::Policy, but provides an
> interface for email-related policies for a specific, restrictive
> environment (e.g. parameters for controlling who can send email, allowed
> recipients, attaching disclaimers and subject prefixes). The DESCRIPTION
> in File::Policy should help explain the rationale behind this.
This sounds like almost (but not quite) entirely unrelated issue, but also
interesting. Start a new thread when you want to talk about it more..?
> :: Software Engineer
> :: BBC Learning and Interactive
It cheers me whenever I see CPAN uploads by BBC. Big corporations that give
back to the free software community make me happy.
--
rjbs
signature.asc
Description: Digital signature
Re: using a "PreferredMailer", unsure about mailer_args.
* Ricardo SIGNES <[EMAIL PROTECTED]> [2006-07-08T10:03:20]
> We have a similar system, actually, but we already implemented "standard
> mailer args" -- for some value of "standard." I've attached a slightly
> bowdlerized version of the module.
Yes, I'm one of those idiots who ALWAYS forgets to attach the message.
--
rjbs
#!perl
use strict;
use warnings;
package Sendmail;
use Email::Send;
use Email::Send::Mailer::SMTP;
use Sub::Exporter::Util;
use Sub::Exporter -setup => {
exports => { sendmail => Sub::Exporter::Util::curry_class('sendmail') },
};
our $default_mailer;
our $mailer_from_env;
sub mailer {
my ($class) = @_;
return $mailer_from_env || ($default_mailer ||= $class->_default_mailer);
}
sub _default_mailer {
my ($class) = @_;
if ($ENV{SENDMAIL_MAILER}) {
return $class->_mailer_from_env;
} else {
return Email::Send::Mailer::SMTP->new({ port => 1025 });
}
}
sub _mailer_from_env {
my ($class) = @_;
my $mailer_class = $ENV{SENDMAIL_MAILER};
if ($mailer_class !~ tr/://) {
$mailer_class = "Email::Send::Mailer::$mailer_class";
}
eval "require $mailer_class" or die $@;
my %arg;
for my $key (grep { /^SENDMAIL_MAILER_\w+/ } keys %ENV) {
$key =~ s/^SENDMAIL_MAILER_//;
$arg{$key} = $ENV{$key};
}
$mailer_from_env = $mailer_class->new(\%arg);
}
sub clear_mailer {
undef $default_mailer;
undef $mailer_from_env;
}
sub sendmail {
my ($self, $message, $arg) = @_;
my $mailer = $self->mailer;
if ($arg->{mailer}) {
$arg = { %$arg }; # So we can delete mailer without ill effects.
$mailer = delete $arg->{mailer} unless $mailer_from_env;
}
Email::Send->new({ mailer => $mailer })->send($message, $arg);
}
"220 OK";
signature.asc
Description: Digital signature
a barrage of releases!
Today, I released new versions of: Email-Address Email-Folder-IMAP Email-Folder-IMAPS Email-LocalDelivery Email-MessageID Email-Simple Email-Simple-Creator Email-Simple-Headers Over the rest of the week, I'll probably release more Email:: bugfixes. Once RT decides that I'm cool enough to close bugs, the total count of Email:: bugs should start dropping. Killing bugs and making it clear, through action, that the Email:: modules are still alive has been my main goal for the last week. By the end of this week, I think we'll have bugs in Email:: down to a manageable level, and I'll be really excited about the prospect of making some progress on new development. I'll probably dump my thoughts here or on the wiki, or both. Expect me to ramble about: * fixing Email::Send * a simpler Email::Send wrapper * a reusable corpus of test messages and an API to it * simplifying Email::Abstract use * standards for mixins * improved usability and ubiquity for Email::Envelope Dump your ideas, too, and we can hash out what we want to do next! -- rjbs signature.asc Description: Digital signature
Re: using a "PreferredMailer", unsure about mailer_args.
* Simon Flack <[EMAIL PROTECTED]> [2006-07-12T16:32:06]
> Ricardo SIGNES wrote:
> >
> > $ SENDMAIL_MAILER=SQLite SENDMAIL_MAILER_db_name=emails.db
> > some-command
>
> I like that, particularly the idea of using mailer_args from the
> environment. It also makes me realise that I'm over-complicating my
> problem.
This project was scheduled to take a chunk of my time, when first approved, and
ended up taking a sliver. That's because, the first day I was to start writing
the code, I got on the bus to work and realized, "I will never use half of
these features." The half I would never use were all the complicated bits.
Now, I'm finding that I want to add back one or two percent of that lopped-off
50%. That means I'll spend another hour or so, which is still a huge win.
I've been able to take the leftover time and start new projects or -- to my
great delight -- write more mailers. You can imagine how much fun it is to
say (to customer support), "Now if you run this program again, but stick this
string at the beginning, all the email will go to your mailstore instead of the
customer's."
> > I submitted a number of patches to Email::Send this past winter, and
> > now that I've taken over maintenance, some of those have been built
> > in. The two that matter, here, are the ability to use mailers that
> > don't start with "Email::Send::" (which the documentation previously
> > claimed was possible, but was not), and the ability to use objects as
> > mailers.
> >
> > This let us create a hierarchy of mailers that DO have a common set
> > of args.
>
> That's a good idea, the only argument that matters to us at the moment
> is how to set the envelope sender address. So I think this is a viable
> solution for us. And if the E::S API changes maybe we can do away with
> those thin wrappers at some point. (The would at least be hidden behind
> E::S::PreferredMailer...)
One stupid thorn in this problem is that we use a base class to avoid lots of
repetition. So, let's say Email::Send::Mailer. Then ::SMTP, ::SQLite,
::DevNull, ::STDOUT, ::TextToSpeech, whatever.
Given how Email::Send finds plugins, it will find Email::Send::Mailer and call
it a plugin. ESM uses virtual methods that die if called ("You didn't
implement this method in a subclass, ninny!") so if ESM ends up going first in
the (unordered) all_mailers used by try_all, things explode.
Sure, it could provide a 0 value for is_available, but then someone will forget
to implement is_available...
So, the better solution is to change the way that plugins are found. I'm just
worried that someone will be relying on their ability to write
Email::Send::Tastes::Like::Chicken and then refer to it as
Tastes::Like::Chicken, or something equally bizarre. (I wonder if recent
patches broke the ability to have a plugin called
Email::Send::Email::Send::SMTP!)
I'm being a pessimist only because I really don't like being at the other end
of the "minor API change breaks your code" stick.
There's always this:
my $sender = Email::Send->new_new(...);
...to get an objet that acts in a civilized way.
> It may be easier to add a new method, and retain backcompat in send().
> E.g. if send() is stuck in time to mean ->send($message,
> @modifier_args), then you could add a $sender->dispatch($message,
> \%mailer_args, \%modifier_args). You would probably want to re-namespace
> the mailers at the same time so old-style mailers won't automatically
> get picked up.
I should've read your whole message before I began to reply! Then again, it's
nice to see someone on the same page.
The question becomes: once we're doing that, what is the benefit of not just
writing Email::Sender? There are now two APIs in one module. As more things
crop up that require differences between ->new_new and ->new, or ->send and
->dispatch, the problem grows. I think it may be simplest to start Sender.
It would be easy, most likely, to write Email::Send::SenderPlugin to serve as
an adapter from Email::Send to Email::Sender.
I am also hesitant to fork something like Email::Send, so I'd cherish more
opinions on the matter.
> I think Return::Value is cute, but it doesn't really fit with the way I
> think. And the following code halts my brain:
>
> my $test = do_something();
> die "$test" unless $test;
I agree. Return::Value was fun to write, and I think it's got some neat ideas
in it, but it has bigger drawbacks even than the sprained brain that can result
above. Case in point:
if (my $value = do_something) {
print $value;
} else {
# failure information not available!
}
So instead you write:
my $value = do_som
wiki pagecount eexplosion
At the suggestion of Dave O, I've populated a stub page for each module in Mail::, Email::, and MIME::. I'm not sure I'm entirely happy about the sheer number of empty pages we have, but I think it might end up being OK. I'll probably write some kind of watcher, later, to update/create pages on new releases... but I'm not sure. I'm also not sure about the NonPEPModules category, simply because I'm not sure what makes a module part of the PEP yet. Is it that the author says it is and the other PEP module authors don't wince? -- rjbs signature.asc Description: Digital signature
Re: wiki pagecount eexplosion
* Ricardo SIGNES <[EMAIL PROTECTED]> [2006-07-12T21:09:41] > At the suggestion of Dave O, I've populated a stub page for each module in > Mail::, Email::, and MIME::. I'm not sure I'm entirely happy about the sheer > number of empty pages we have, but I think it might end up being OK. Someone suggested that I should've included a linke: http://pep.pobox.com/wiki -- rjbs signature.asc Description: Digital signature
Re: wiki pagecount eexplosion
* Dave O'Neill <[EMAIL PROTECTED]> [2006-07-12T22:02:31]
> Sheer volume aside, it's probably still a good idea. I don't think
> there's a single-page comprehensive listing of email-related modules
> anywhere else.
I know there are other modules that fit in. There's the SMS and MMS series...
suggestions welcome.
> Maybe it doesn't make sense to keep that category. Given that there
> would likely be about a hundred or so modules in it, it's probably
> easier to mark the PEP modules as such (whatever we determine that to
> mean), rather than marking non-PEP modules.
I think that's a good idea.
> Earlier today we were talking on IRC about some possible categories. I
> don't recall quite what was said, but I think the potential starting
> categories were something like this:
> - all CPAN modules that send, receive, parse, or manipulate email
> (basically, Category:Modules as it exists now)
> - modules owned/maintained by PEP
> - modules that PEP recommends (both PEP-maintained and
> not, will change over time)
> - modules that PEP recommends you not use (unmaintained,
> deprecated, or just plain bad)
I think this is an accurate portrait of what we talked about. I think the
relevant categories will be:
Modules - it's a Perl module!
PEPMaintained - it's in the PEP repository and will follow PEP guidelines
(yet to be determined; things like, "n people review API
changes")
PEPEndorsed - x out of y active PEP contributors think this is a good
module for what it does
SeemsAbandoned - no updates in a long time, despite open bugs
Superseded - another module provides all of the functionality, better
I've also created these categories:
HasProblems - the article includes a description of problems that go beyond
simple-to-fix bugs; either they're hard or they're design
issues
HasIdeas- the article includes a section about ideas for the future
The latter two should be created with templates: {{Problems}} and {{Ideas}}
will create a section header and category marker.
> At this point, it probably makes sense to define a "PEP module" as the
> email-handling modules that are a) authored or currently maintained by
> someone participating in PEP and b) don't make other PEP authors wince.
> If that turns out to be unworkable in the future, we can always
> change our minds, but we have to start somewhere.
Works for me!
I'm going to JFDI and stick these categories where I think they belong. Argue
or extend my ideas as y'all see fit!
--
rjbs
pgpJwHSX7zQeb.pgp
Description: PGP signature
more wiki organization thoughts
First off, I should note that I made it through a lot of the Email:: modules last night, adding categories, descriptions, problems, and ideas. It's all very sketch-like and not very blueprint-like, but I think it's a good start. We're up to 200 modules, by the way, and I haven't even loaded all of Email::Store. I think we might want or need a way to mark some modules as ancillary -- the page for Email::Folder::Maildir does not need to be listed in as many places as Email::Folder. There are some dists with even less important ancillary modules. In fact, though, I think we want to list things the other way: rather than say that Email::Folder::Maildir is ancillary, I would say that Email::Folder is priamry. That's because I'm likely to want a listing of all primary modules, but not all non-primary modules. Every dist should have one or more primary modules: they're the ones you're likely to "use" in your code. ...immediately after drafting this, I went to the wiki to JFDI. I realized that many modules that have their own dist are parallels to what I would have otherwise called support modules. Email::Folder::IMAPS is its own module, but is parallel to Email::Folder::Mbox, which is a supporting tool in Email-Folder. Was this idea a poor one? -- rjbs pgpV4l2NNLIFE.pgp Description: PGP signature
Email::Abstract - new version, new problems?
So far, we've closed around fifty outstanding bugs in PEPMaintained code. We
deserve a beer. (Hm; note to self: patches == beers at next PEP gathering.)
With most of the PEP bugs closed, I feel good about starting to work on
improvements. I thought I'd avoid much controvery and annoyance by starting
with the relatively innocuous Email::Abstract, but I was annoyed and feel like
the solution is to do something controversial. Oops!
The only-static-method-calls interface to Email::Abstract has always seemed
needlessly wordy to me. I don't want to type this:
for my $message (@messages_of_unknown_class) {
if (Email::Abstract->header($message, 'Subject') =~ /urgent/) {
Email::Abstract->header_set(
$message,
'Subject',
'[URGENT] ' . Email::Abstract->header($message, 'Subject'),
);
Email::Abstract->header_set($message, 'Priority', 'high');
}
}
Instead, I'd rather type:
for my $message (@messages_of_unknown_class) {
my $email = Email::Abstract->new($message);
if ($email->header('Subject') =~ /urgent/) {
$email->header_set('Subject', '[URGENT] ' . $email->header('Subject'));
$email->header_set('Priority', 'high');
}
}
I think that's a pretty reasonable desire, and pretty easy to implement. It's
implemented and tested in Subversion, and I'll probably release that soonish.
(I need to fix an undocumented and potentially unmanifested bug related to
multiple points of plugin/inheritance matching, first.)
Email::Abstract has a bunch of plugins that adapt individual classes. For
example, there's Email::Abstract::MIMEEntity that provides the wrapper for
MIME::Entity. I thought the simplest thing to do would be to bless
Email::Abstract->new's return value into Email::Abstract::Object, which would
be isa the base type for Email::Abstract plugins.
The first problem: there is no base class for Email::Abstract plugins. In
fact, the documentation says:
Other representations are encouraged to create their own "Email::Abstract::*"
class by copying "Email::Abstract::EmailSimple". All modules installed under
the "Email::Abstract" hierarchy will be automatically picked up and used.
So, let's forget about the fact that the documentation gives the dangerous
instruction of copying code and just decide that no base class is needed. We
just need to implement the same methods.
If we do this and write Email::Abstract::Object, though, we've introduced a new
problem: Email::Abstract will always try to use a predictable plugin name for
any given object. That means that if we pass an object of class Object to the
class method "header" on Email::Abstract, it will be sent to the
Email::Abstract::Object's methods. First of all, those methods would have to
be polymorphic (to accept a message or not as the first argument) to even
pretend to work. Secondly, they'd fail anyway, since Email::Abstract::Object
doesn't exist to handle Object objets.
We can't move Email::Abstract::Object to another namespace under
Email::Abstract, either, because Email::Abstract assumes that any module under
the Email::Abstract namespace is a plugin. (It also assumes that nothing else
ever will be.)
Ok, so, this ends up being not a big deal for this particular problem. I added
the polymorphism to Email::Abstract itself, and bless objects right into that.
The problem is that this can strongly negatively affect future module authors.
What if you want to write Email::Abstract methods for dealing with MIME parts?
You can't write Email::Abstract::MIME, because that would handle MIME objects.
I see a few potential solutions:
1. Email::AbstractX:: for non-plugin support modules.
2. an optional is_wrapper method which, if it returns false, causes the
module to be excluded from the list of plugins; maybe is_wrapper is just
the current "target"
3. plugins should be moved to Email::Abstract::Wrapper::
I think that option 1 is ugly. Foo-x namespaces seem to me like a scar that
permanently shows where you made a design mistake. (Another version of this
solution is use Email::Abstract:: but to prefix all third-part names with a
numeral, so they could not possibly represent valid class names.)
Option 2 is also a permanent scar, but it's a subdermal one; it hides the
mistake under the surface, where only developers of Email::Abstract extensions
need to worry about it. Even that's pretty lousy, I think, and can still end
up confusing: it means that Email::Abstract::EmailSimple and
Email::Abstract::Simpleton are externally indistinguishable, although one
is_wrapper and the other is not.
I vote for option 3. The problem is that if people out in the world have
written their own Email::Abstract plugins, they will be ignored. A partial
solution is to go through a deprecation period during which non-core plugins
are still loaded, but a warning is emitted. I am not excited about the fact
that this could still break someone's code, especially if he upgrad
recent releases
This week, I released a new version of Email::Abstract, featuring the object-based interface I described. So much less typing! I think I'll be using Email::Abstract a lot, now. I also released Email::Send 2.10, featuring a number of the changes I'd mentioned some time ago. I think my next set of changes will be against Email::Store, which has a good few bugs built up. Volunteers welcome. ;) -- rjbs signature.asc Description: Digital signature
svnnotify (get email on commits)
Tonight, I changed our Subversion notifications from emailing "me and Dieter" to send the mail to a mailing list. You can subscribe to it by sending a message to [EMAIL PROTECTED] or by going here: http://listbox.com/subscribe/?listname=pep-checkins -- rjbs signature.asc Description: Digital signature
open bugs!
I've been feeling sort of sleepy and uninspired, so I decided to do a little yak shaving. I've been a little annoyed at the fact that it's not trivial to ask rt.cpan.org for "all the bugs attached to dists that I maintain," so I finally wrote something to do it for me. Then I ran it against all the Email, Mail, and MIME dists. Attached, find a list of all the open bugs against Email, Mail, and MIME dists. It's surprisingly short, but I don't /think/ that's a bug in the bug aggregator! -- rjbs id| sev | dist | subject | update 4483 | Crit | Mt | MIME::Parser::Reader ; can't call method "isa" | 2 ye 17654 | Crit | MF | Cannot compile MIME::Fast | 5 mo 19655 | Crit | MLH| include_css is far too aggressive | 7 we 2803 | Impo | Mt | deprecated MIME::Head::decode() has been rem... | 3 ye 2760 | Impo | ML | send_by_sendmail() chooses incorrect default... | 3 ye 6115 | Impo | Mt | MIME::Head recommend_filename doesn't handle... | 2 ye 5595 | Impo | ML | MIME::Lite->new add 2 characters '! ' to the... | 2 ye 3968 | Impo | ML | 5.5.4 invalid address with send_by_smtp only| 2 ye 2759 | Impo | ML | send() doesn't set sender with send_by_sendm... | 2 ye 13027 | Impo | Mt | MIME::Words::encode_mimewords split one char... | 1 ye 7368 | Impo | Mt | STDERR print statements WordDecoder | 1 ye 8663 | Impo | Mt | MIME::Decoder::NBit changes only some \r\n t... | 1 ye 7457 | Impo | Mt | binary data incorrectly encoded into quoted-... | 1 ye 7709 | Impo | ESto | French characters in subjects not parsed cor... | 1 ye 13082 | Impo | ML | Line endings are wrong: test 4 in t/data fails | 1 ye 8342 | Impo | ML | "make test" failed in t/data test 4 | 1 ye 12264 | Impo | ML | Some Japanese mobile phone email addresses c... | 12 mo 17827 | Impo | ML | boundary not set for $mail_msg->print() | 5 mo 17319 | Impo | ML | Timestamp issue | 3 mo 19165 | Impo | MBase6 | Compile on HP-UX B.11.23 ia64? | 2 mo 19500 | Impo | MLH| cid has no effect on background images | 5 we 13236 | Impo | EMI| No MIME-Version header field set, but RFC 15... | 2 we 20474 | Impo | Mt | require MIME::Parser eats DATA section of ca... | 8 da 3289 | Norm | ME | File attachments with semicolons in their na... | 2 ye 7108 | Norm | EMAS | Unable to strip the attached message - 1| 1 ye 11901 | Norm | Mt | Can't call method "remove_sig" on an undefin... | 1 ye 8512 | Norm | Mt | ... or die won't work in Parser | 1 ye 8101 | Norm | Mt | MIME::Parser::Reader removes trailing \r fro... | 1 ye 7858 | Norm | Mt | MIME::Parser::init_parse() partially thwarts... | 1 ye 13867 | Norm | EMAS | Unable to get body for parsed message from a... | 12 mo 13854 | Norm | EStr | header parsing: message string inconsistent | 12 mo 13887 | Norm | EAd| Quoting the Phrase | 12 mo 14087 | Norm | EST| setup x2 will croak | 12 mo 15283 | Norm | ELE| Should chmod messages | 9 mo 15512 | Norm | ML | typo in send_by_sendmail() method | 9 mo 16528 | Norm | MBase3 | Encode/decode trash $_ | 7 mo 17188 | Norm | ML | Headers with \n wreak havoc | 6 mo 11452 | Norm | Mt | MIME::Parser: can't flush | 5 mo 18059 | Norm | Mt | Weird utf8 bug breaks header parsing? | 4 mo 14076 | Norm | EMCo | Blank content-type attributes spew warning | 2 we 18805 | Norm | ESi| header method return values | 2 we 13091 | Norm | ESto | Enhancement to let Email::Store::Mail->store... | 6 ho 7215 | Unim | ML | $SENDMAIL not defined on Win32 | 1 ye 16320 | Unim | EAd| processing address with many white spaces | 11 da 6789 | Wish | Mt | MIME::Parser::Filer evil_filename suggestions | 2 ye 18847 | Wish | EMM| Content-ID | 2 we 19733 | Wish | ESe| allow per-message args to mailer| 2 we 437 | | ML | MIME::Lite 2.106-2.117 | 4 ye 6814 | | EMI| parts_multipart not able to parse discrete t... | 2 ye 7841 | | MLH| Text-Only Encoding Ignored | 1 ye 13375 | | EE | Small typo in method name | 1 ye 12784 | | Mt | Clarify dependency on IO::InnerFile 2.110 | 1 ye 9668 | | ML | Module overrides global PATH environment var... | 1 ye 8558 | | EFil | Email::Filter-based scripts need to be easie... | 2 we 20440 | | EAu| promoo subma
obsoleting Email::Simple::Headers?
Part of the idea behind the Email:: modules was that they each did one thing
simply and well. For example, Email::Simple turned a RFC822-message-like
string into an object representing that message. It didn't let you build such
a message without a string to start from. If you want to do that, you use
Email::Simple::Creator.
That strikes me as pretty reasonable, and with some improvements to how the
pieces fit together, it will make for componants that can be used for all
manner of useful things.
That said, I think Email::Simple::Headers should be retired. It provides one
method, implemented in one line of code:
sub headers { values %{ $_[0]->{header_names} } }
Unarchived, its dist takes up 40K.
Now, this would be less obnoxious if it weren't for something else: someone --
LTJake, I believe -- recently brought to my attention that there is no easy way
to get an ordered listing of each headers and values. That is, not only is
there no method, but there is no way to do it without violating encapsulation.
While looking into this, I found that stringifying a message was very likely to
reorder its headers and fixed that. In the course of fixing that, I had to
implement the feature LTJake wanted, internally, in the _headers_as_string
method.
So, now a very basic Email::Simple method relies on a feature that would seem
to belong in Email::Simple::Headers. Since it seems like it would be silly to
have a circular dependency, or to bundle the modules, I think that we should
acknowledge that Email::Simple::Headers is silly and move on.
I'll put E::Si::H into Email-Simple and have it do nothing. I will move the
headers method to Email::Simple, and will add a way to get headers in order,
with duplicates. (Right now, $email->headers returns only unique, unordered
header names.)
This will probably involve:
making ->header($name, $i) return the $i-th value.
writing a ->header_pairs method that returns ($name1, $value1, ...)
Doing this in Email::Simple should make it easier to implement more of
Email::Simple in terms of its own methods, which will be a win.
If you object, please speak now, or forever hold your peace!
--
rjbs
pgpInxNZhSdns.pgp
Description: PGP signature
Re: possible changes for Mail::DeliveryStatus::BounceParser
* William Yardley <[EMAIL PROTECTED]> [2006-08-03T18:09:01] > I've also been working at building some tests for some of the problems > we've seen, and building a bigger corpus of emails to use for testing. I am really looking forward to having a nice body of messages selected for use as proof that feature X is any good. > I have most recently been working on messing around with the regexes for > "user_unknown" (see changes around line 796), and using 5.1.0, 5.1.1, > 5.1.2 and 5.2.2 errors[1] in the status report as a preferred method of > determining $report->std_reason over text regexes (see changes around > line 374). Excellent; I would much rather trust the data that's supposed to tell us something than the "data" whose entrails we have ripped out read. > I also ripped out some AOL / Hotmail specific hacks which I'm pretty > sure are way out of date now. I bet there are more yet to go. > http://veggiechinese.net/bounce_parser_diff_3.txt > > None of these changes are checked in; at this point, I'm just soliciting > opinions, and hoping some other folks might be willing to test these > changes. I'd suggest you check them in; it'll be easier to test them. Consider making a branch. Looking at the code changes, I think checking them in would be good. They're definitely an improvement, unless I'm missing some obnoxious bug. > I made a few suggestions (the last 2) at: > http://emailproject.perl.org/wiki/Mail::DeliveryStatus::BounceParser "bounce unless otherwise" is, yes, dumb. I think that leads to a lot of grief. It makes sense in the project's original context, when it was known that it should only be seeing bounces, and the author wanted to be able to exclude some things. In production, it's an obnoxious assumption. It should be easy to make that an option. I think I'll be deploying these changes, after a bit more testing; I hope to see some improvements in junk avoidance. -- rjbs signature.asc Description: Digital signature
Re: possible changes for Mail::DeliveryStatus::BounceParser
* William Yardley <[EMAIL PROTECTED]> [2006-08-03T23:25:31]
> > Excellent; I would much rather trust the data that's supposed to tell
> > us something than the "data" whose entrails we have ripped out read.
>
> Yeah. The only problem is what to do when there is some sort of conflict
> between the two bits of data. In theory, you'd hope that people
If we find examples of this, the behavior can be tailored to those specific
examples, rather than remain so general. I think this is still a win.
> I've been thinking about writing a "blocked" std_reason for bounces that
> appear to come from spam or virus filters, or other site policy type
> blocks - this might make dealing with those blocks a bit easier for
> organizations... even the most legitimate of lists end up getting
> blocked by someone, somehow.
Sounds good.
> You mentioned something about "no_problemo" - maybe we should get rid of
> it entirely?
Well, I wish it was something like not_a_bounce, but we can't just change it,
as things probably rely on it. Adding code to make MBP return something
else based on an option seems like it could introduce bugs with no clear
benefit (beyond feeling less silly when checking std_reason). What we *can* do
is only return it when std_reason is called on a parsed non-bounce, rather than
actually setting it on the object. So:
sub std_reason { return $_[0]->{std_reason} || 'no_problemo' };
(We can make 'get' use methods to get internal values, to make this work. This
lets us provide more dynamic behavior, which is good. 'get' is a dumb method.)
That would also let us drop ->{is_bounce} and have the is_bounce method return
the std_reason, leaving it undef for non-bounces Same behavior, more
information.
--
rjbs
>
> w
>
signature.asc
Description: Digital signature
recent activity
Well, life got pretty busy lately, and I got a little lazier. This was a bad combination, but I'm back on track! Here's what's been going on with and without me in PEP land: WBY has been committing a constant stream of little improvements to the BounceParser, which is slowly going from pathological and nearly unusable to pathological and fairly effective. We'll probably see another 1.5xy release this week. BRICAS and SSORICHE (and maybe others who I can't presently recall) committed some improvements to Email::Simple and ::MIME modules, and I finally made an update I've had on the TODO for a long time: ->header($x) now returns undef if there is no $x header. I expect a lot of people to start seeing warnings, but I think that this is a price worth paying if it means that you can easily tell that the header you asked for didn't even exist. I continued my role of "jerk who keeps bugging CPAN authors about the bugs that don't bug them" -- I'm getting pretty good at it. I think pretty soon people might see me in their inbox and immediately start grumbling. (That's a good thing... isn't it?) As an upshot, there's a new MIME::Lite::HTML release that fixes a few critical bugs. If you use MIME::Lite::HTML, you could let Alian know that you are happy to see it still alive. It might make up for all the "wtf dood!" emails that I pestered him with while he was on vacation. Also, MIME::Lite itself will soon be coming to the PEP repository. It is, I think, one of the modules in which bugs have amassed. Yves is looking forward to having PEP contributors smother it with bug fixes, and I am looking forward to a decrease in the overall bug count for Mail/MIME/Email modules! Here's what I see in PEP's future, from my end: * another pass through the PEPMaintained modules for bug fixes * some time spent helping MIME-tools modernize their test suite * a first run at interfaces for a new email sending module * a first run at an implementation of Email::Corpus * an afternoon of Email::MIME testing, with lots of RFC consultation Hopefully with those done I can start getting to the things I really want to write for fun. How's everyone else out there in the emailverse doing? -- rjbs signature.asc Description: Digital signature
Re: RFC compliance
* Alex Efros <[EMAIL PROTECTED]> [2006-09-13T09:05:50] > Here is short summary - is there exists email parser which comply to all > (or most) email format related RFC: > 1344 1847 1864 2015 2045 2046 2047 2049 2183 2231 2822 3156 I really doubt it. Let me know if you find out. On the Monastary, you say, "defect free." If you include that, then no. There is no non-trivial software that has no bugs. > For example, it should be able to correctly handle multipart/mixed, > multipart/related, multipart/alternative, message/partial, etc. MIME types and > all possible formats of email addresses, comments in email header fields and > all things like locale/language/encoding-specific, for example: The best way to find out whether any software can do these things, or to check it against the RFCs, is to test it. If you were to write a fairly portable set of tests to ensure compliance with the above RFCs, you would be greeted with great plaudits and probably at least a few beers. As for Email::, most of its code was intended to be strict in what it produced and loose in what it accepted. In reality, it is not perfect. More tests would be adored. Tests that could be easily ported to Mail::Internet and MIME::Entity would also be fantastic. -- rjbs
Re: Mail::DeliveryStatus::BounceParser - qmail support?
* Ricardo SIGNES <[EMAIL PROTECTED]> [2006-09-26T07:37:13] > * William Yardley <[EMAIL PROTECTED]> [2006-09-26T01:09:20] > > Oh crap! Doesn't look like *any* of the new or updated tests from SVN > > are in the tarball in CPAN. Paging rjbs to the red courtesy phone > > Fixed in 1.516. I'll release that once I get to an internet spigot. Bah, released with changelog uncleanedup. Well, that's life. -- rjbs
Re: Mail::DeliveryStatus::BounceParser - qmail support?
* William Yardley <[EMAIL PROTECTED]> [2006-09-26T01:09:20] > Oh crap! Doesn't look like *any* of the new or updated tests from SVN > are in the tarball in CPAN. Paging rjbs to the red courtesy phone Fixed in 1.516. I'll release that once I get to an internet spigot. -- rjbs
MBP: orig or final recipient?
This code appears in MBP's parse method:
374next unless my $email = $report->get("original-recipient")
375 || $report->get("final-recipient");
In other words, if we get some DSNs, and they give an Original-Recipient, use
it as the email that bounced. Otherwise, use Final-Recipient.
This code was originally written to handle bounces to an exploder (an MLM), so
this behavior strikes me as wrong. If a messages goes to the exploder and
bounces at its target, the DSN will look like this:
Original-Recipient: [EMAIL PROTECTED]
Final-Recipient: [EMAIL PROTECTED]
Well, argh! Why are we going to then say that the mailing list explosion
address bounced?
I really want to change this to reverse the order, but I'm afraid that someone
out there who is NOT using this for mailing list bounces, or who is concerned
about exposion further down the line is going to be affected by this.
There are a few choices:
a. do nothing -- this doesn't work for me
b. reverse the order in the code -- great for me, how about you?
c. make it an option -- ugh, more options!
z. fork it internally -- utter last resort
Am I being paranoid about (b)? MBP is in that class of module where it has
been so lousy for so long thatn it's odd to think that many people are relying
on it. Then again, I know people are -- I just don't know which brain-dead
behaviors they rely on, and which they just cope with.
--
rjbs
Email::Simple header developments
Email::Simple 1.992 fixes a bug introduced in 1.96, in which headers could not always be added with more than one value. This bug was introduced when we made header ordering reliable. The previous code only ensured that each header had one entry for ordering, as it output all headers of a given type in a block. Clearly, Email::Simple needs improved testing, although I've added a good bit in response to this problem. This problem manifested when I was trying to add a "prepend_header" method to allow header_set-like behavior that would stick a header pair at the start of the headers. The lack of this method, or of other more sophisticated header handling, has made me want to get back to making Email::Simple easier to subclass and extend. Right now, Email::Simple subclasses, like my ES::FromHandle, rely on using private meth^Wfunctions from Email::Simple. This is clearly horrible. Instead, we need to document and make semi-public more of E::S's internal routines for things like "turn a multi-line header block into a list of pairs." The latest trunk of Email::Simple takes a few baby steps in this direction, breaking header building up into bits and adding a private header object that actually implements the header methods. If anyone has thoughts on what bits need to be more public and documented, I'm eager to hear them. I know _split_head_from_body is something I had to copy into FromHeader, which stinks. All or much of the new headers stuff should be public. -- rjbs
Email::MIME::CreateHTML
The BBC, peace be upon them, uploaded Email::MIME::CreateHTML this week. I'm very excited! It replaces MIME::Lite::HTML, which has seemed to me a less than perfectly maintainable module. I haven't started using it yet, but I'm hoping I can replace my MLH use with it pretty soon. -- rjbs
Email::Simple memory wrangling: testers wanted
Over the last few days, I've done a bunch of memory wrangling on Email::Simple, as well as other refactoring. The upshot is that dealing with large emails takes drastically less RAM. In my tests, about half as much. While an eight meg message took about 40 MB (beyond the original text) to remain resident in 1.996, 1.997_01 takes between zero and eight megs. For details, you can consult recent checkins, or my comments in my use.perl.org journal, here: http://use.perl.org/~rjbs/journal/31737 http://use.perl.org/~rjbs/journal/31749 I'd appreciate tester feedback on the development build, which is now in the incoming queue on the CPAN. My biggest concern is the fact that only the latest releases of Email::MIME will work with this Email::Simple as their base class. Earlier revisions were too eager to muck about with the Email::Simple-created guts, which have changed. This concerns me because the "obvious" solution is something like this: if the new Email::Simple is installed, and an old, incompatible Email::MIME is installed, require a newer Email::MIME first. That might work, but this Email::Simple adds a ->crlf method, which Email::MIME will want to use in the future, which means that it will want to add a higher version prerequisite for Email::Simple. This will create a circular dependency, which I'm sure will cause problems. Is this a solved problem, and I just don't know about it? Enjoy! -- rjbs
Re: Net::SMTP_auth SSL/TSL
* Justin Simoni <[EMAIL PROTECTED]> [2007-01-17T23:20:12] > Does anyone think it would be as easy to make a, Net::SMTP_auth::SSL > module? My first glance at the code for Net::SMTP_auth and Net::SMTP::SSL makes me think that it should be quite easy. A Net::SMTP_auth::SSL isa Net::SMTP. I'd like to think you could make it be a Net::SMTP::SSL and that would be that. The package-copying code in Net::SMTP::SSL should be pretty easy to copy. It sucks that you need to do it, but it's easier (or at least saner, I think) than trying to use something else to alter the method dispatch order. Let us know how it goes. -- rjbs
Re: Can Email::Simple parse all emails
* abhishek jain <[EMAIL PROTECTED]> [2007-01-19T13:32:05]
> I went through the documentation of Email::Simple and it says it parses
> RFC2822 emails , does this covers all the emails , if not is there a module
> which gives me the contents of the emails(subject and body) as a text, if
> not tell me where to start from.
Email::Simple should be fine.
my $email = Email::Simple->new($text_of_message);
my $subject = $email->header('Subject');
my $body = $email->body;
--
rjbs
Re: trouble installing Email-MIME-1.857
* Jim Graf <[EMAIL PROTECTED]> [2007-02-09T16:56:35] > I wanted to try Bugzilla 2.23.4 and they now require Email::Send, which > requires various Email:: modules, including Email-MIME. Yup, I'm aware of the problem, and totally unsure of how this happened. Well, that's not true. I know it happened because I prepared the release ages ago and only made it just the other day. I'm going to get this sorted out before I got to bed, even if it means releasing 1.996 as 1.989... watch the skies. Sorry for this. :-/ -- rjbs
Re: trouble installing Email-MIME-1.857
* Ricardo SIGNES <[EMAIL PROTECTED]> [2007-02-09T21:24:36] > * Jim Graf <[EMAIL PROTECTED]> [2007-02-09T16:56:35] > > I wanted to try Bugzilla 2.23.4 and they now require Email::Send, which > > requires various Email:: modules, including Email-MIME. > > Yup, I'm aware of the problem, and totally unsure of how this happened. Well, > that's not true. I know it happened because I prepared the release ages ago > and only made it just the other day. Email::MIME 1.858 has been uploaded. The problem is something like this: Email::Simple 1.998 has an improved header object and parser, which turns out to be a smidge stricter than Email::Simple 1.996. Email::MIME may pass in the text for the sub-parts of a message something like "\n\nactual header..." The leading whitespace caused the header to be parsed improperly. That whitespace is now stripped. Email::Simple 1.998 is a big win for memory usage, so I really didn't want to roll it back. I'm glad I didn't have to. Please let me know if you encounter any more problem with these releases. -- rjbs
Re: Email::Simple changes?
* "Karen J. Cravens" <[EMAIL PROTECTED]> [2007-02-14T12:34:12]
> I'm not sure if it's Email::Simple, Email::LocalDelivery, or me (it's a
> new server, continually being tweaked, so something else might have
> changed), but suddenly (that is, three days after E::S1.998 was installed
> and running fine) this:
>
> my $delivery_agent = Email::LocalDelivery->deliver($message->as_string,
> ("/blahblah/blah.mbx"));
Aaaauuugh, argh, grash.
Thanks very much for your bug report. Here is the problem:
sub _escape_from_body {
my ($class, $mail_r) = @_;
# breaking encapsulation is evil, but this routine is tricky
my ($head, $body) = Email::Simple::_split_head_from_body($$mail_r);
$body =~ s/^(From\s)/>$1/gm;
return $$mail_r = "$head\n$body";
}
Well, sir, just because things are tricky doesn't mean you get to break the
three laws of robotics!
Anyway, 0.211 is now on its way to the CPAN, and replaces that code with
something with a fully public interface.
You know, the problem here is that Email::Simple originally had code useful
enough that lots of things wanted to use it -- LocalDelivery, FromHandle, MIME
-- but it didn't make it public, so this kind of thing happens. I guess
because most of this code had one or two authors, there was some sort of belief
that the internals would never change.
Oh well! Please update to 0.211.
--
rjbs
Re: Email::Simple changes?
* Ricardo SIGNES <[EMAIL PROTECTED]> [2007-02-14T16:57:49] > Oh well! Please update to 0.211. Update to 0.213. 0.211 and 0.212 could have problems when delivering the same message to multiple destinations, which was not properly covered by the test suite. -- rjbs
Re: Email::Simple changes?
* "Karen J. Cravens" <[EMAIL PROTECTED]> [2007-02-14T17:50:27] > Yeah, errm, I've actually got some code that uses the internals directly > myself, that I'm gonna hafta change before it gets into production. > > In fact, I think I *use* _split_head_from_body, now I think about it. > I'll have to look at it. It was just supposed to be temporary. > > Perhaps it's useful enough to make it public? In the past, I've mumbled something about making Email::Util for this and some other things (like folding or unfolding long headers). Maybe I should get around to it. -- rjbs
Email::Send Redux
So, I've mumbled about problems in Email::Send a number of times in the past, and I already "fixed" a number of them to my own satisfaction in the fork of Email::Send that we use at work. Unfortunately, it's not quite backwards compatible (if you do weird things, but I'm sure people are doing weird things.) Also, it has to go through some silly hoops even to be as compatible as it is. I've talked about writing a totally incompatible replacement, and I've more or less done that this week (although mostly I've just stripped down my forked version and rebuilt it from its parts). I'll post the code shortly, but first I'm just going to talk about the design. I'm tentatively calling this module Email::Sender. Email::Send has two important classes. One is Email::Send. An object of that class (a sender) accepts a message, makes sure it's an Email::Simple, and then passes it along to the other kind of thing. That other thing is a mailer. A mailer has to implement two methods: is_available is a predicate that answers the question, "can I send a message with the given mailer args?" and the send method attempts to actually send a message. There is no common base class for mailers. Their interface is about as concrete as I've stated above, with the added constraint that "send" must return a true or false value. Many of the included mailer classes use Return::Value for return values. Email::Send, if not told what mailer to use, will try them all in a random order and stop when one works. Because the arguments used by mailers is not standardized, the "try all" behavior more or less cannot be used with mailers that have args. Okay, that's enough about Email::Send. Email::Sender has one main class, the sender. Email::Sender's send method accepts a message and arguments, canonicalizes the arguments, and tries to deliver the message. It returns either a "successful delivery" report or raises an exception. Specific transports are implemented as subclasses of Email::Sender. Most will only override the (virtual) "send_email" method that does the sending. It will always receive an Email::Simple-like object and arguments including "to" and "from" to be used in transporting the message. Because the return value of a successful delivery always has a known interface (currently I'm calling it Email::Sender::Success, but something more like Email::Delivery::Successful seems better, but also silly), and because there is at least a small core of guaranteed common arguments, senders can be swapped out with a known worst-case level of degredation. This makes it easy to write two useful pieces of code: (1) Email::Sender::Multiplex, which acts like Email::Send's try_all mode, but can be limited to certain senders with known arguments and in a known order (or could easily be told to just find everything and use default arguments, if you're feeling stupi^Wlucky). (2) Email::JustEffingSendIt, which accepts -only- the minimal core arguments. It uses some default sender, but calls to its single ->send method may pass a "sender" argument. Further, if some PERL_EMAIL_SENDER env var is set, along with a cadre of related vars, they will globally enforce another sender so that programs can be run in test mode, directing all their mail to SQLite or a local Maildir or /dev/null. I have found this technique (which I have already implemented at work) to be of nearly endless value. I haven't worried much about the "message modifier" feature of Email::Send, which I'm guessing is not very popular, and could easily be added later or by something like Email::Sender::Wrapper (which also exists and adds triggers to the sending process). I am eagerly looking forward to questions, comments, praise, or snide remarks. -- rjbs
Re: Email::Send Redux
* Dave O'Neill <[EMAIL PROTECTED]> [2007-02-21T00:07:06]
> On Tue, Feb 20, 2007 at 07:07:43PM -0500, Ricardo SIGNES wrote:
>
> > Email::Sender has one main class, the sender. Email::Sender's send method
> > accepts a message and arguments, canonicalizes the arguments, and tries to
> > deliver the message. It returns either a "successful delivery" report or
> > raises an exception.
>
> There's already a Mail::Sender on CPAN... is Email::Sender enough of a
> name collision to cause confusion?
I'm not sure. I also don't like Email::Mailer, for the same reason.
Email::Delivery::Agent would be okay, but is pretty long. Then again, if most
people use the simplified interface (like I do, in effect, at work) it may not
be so bad. Also, it leaves Email::Delivery::Success and so on free.
> Are you planning to define exception classes, or just leave it as a
> free-form die q{Failed for some reason};?
Probably classes. That's actually what our version at work does.
> might be nice for error handling and recovery. But, it might also be well
> outside the scope of what you want to do with Email::Simple / Email::Sender.
Right. I'm not sure how inclusive I want to be yet.
> Providing an envelope sender and recipient(s) separate from the From/To
> headers of Email::Simple is definitely a good idea. Maybe we need an
> Email::NotQuiteSoSimple (bad name, but just for discussion's sake...)
> that's basically an Email::Simple with envelope_sender() and
> envelope_recipients() methods added. They could either be set directly,
> or if left unset, would pull their values from the appropriate
> $self->header('...') values.
We're working on something like this at work. The idea is something like:
There is a class for emails, a class for envelopes, and a composite class. I
don't think Email::Envelope is what we want for the envelope, but I'm not sure.
The current behavior in Sender is to fall back to the to/from headers if not
given explicitly. With an Email::EmailWithEnvelope, you'd fall back to the
envelope sender/rcpt if no explicit one was given, presumably.
You'd never let your envelope "fall back" to the email's values, because
they're not related.
> > Because the return value of a successful delivery always has a known
> > interface (currently I'm calling it Email::Sender::Success, but something
> > more like Email::Delivery::Successful seems better, but also silly), and
> > because there is at least a small core of guaranteed common arguments,
> > senders can be swapped out with a known worst-case level of degredation.
>
> How about Email::Status::Success? Email::Status could be an abstract
> base class for the return codes and exceptions as well, specifying a
> common API for getting/setting error text and SMTP error codes.
Hm. Sounds extremely top-level and vague. That could be good or bad. :)
--
rjbs
Re: Email::Send Redux
* Hans Dieter Pearcey <[EMAIL PROTECTED]> [2007-02-21T10:26:12] > On Wed, Feb 21, 2007 at 10:08:50AM -0500, Ricardo SIGNES wrote: > > I'm not sure. I also don't like Email::Mailer, for the same reason. > > Email::Delivery::Agent would be okay, but is pretty long. Then again, if > > most people use the simplified interface (like I do, in effect, at work) it > > may not be so bad. Also, it leaves Email::Delivery::Success and so on > > free. > > 'Delivery' makes me think of receiving/endpoints rather than origination. I am not sure that this matters! Why is Email::LocalDelivery not implemented in terms of Email::Send? Lack of vision I say! :) You generate an email and send it to your MTA destined for me. You use Email::Delivery::Agent::SMTP. My MTA sends it to my MDA, which writes it to my spool using Email::Delivery::Agent::Maildir. I'm not sure there's any difference beyond how it gets transported to its destination. -- rjbs
Email::Simple::Creator, false headers
I thought I'd have a quick run through Email::Simple::Creator to clean up some
of its foibles. Here is another one that makes me wonder...
False Headers
Remember _add_to_header?
sub _add_to_header {
my ($class, $header, $key, $value) = @_;
return unless $value;
${$header} .= join(": ", $key, $value) . $CRLF;
}
That return unless $value is nuts! It means that you can't create a message
with "0" or "" as a header value. This is clearly a bug.
The problem is that unless a Date header is specified, one is generated and
added. Currently if one specifies a false Date header, nothing is added,
either the given (false) value or a generated value.
The code below creates a message with one header, Subject:
my $email = Email::Simple->create(
header => [ Date => undef, Subject => 'foo' ],
body => "Hello sailor.",
);
If the bug is fixed in the way that seem obvious to me -- use '' for empty and
undef headers and 0 for literal zero headers -- then people using constructs
like the above will now have Date fields again -- they'll just be blank.
I don't really like the idea of "undef is skipped but '' is not."
I am sort of leaning toward special-casing Date. It sucks, but it's already a
special case to begin with.
--
rjbs
Email::Simple::Creator
I thought I'd have a quick run through Email::Simple::Creator to clean up some
of its foibles. Here are the ones that make me wonder...
The changelog says:
1.3 2004-07-05
Create a message using its local crlf (Casey West).
Include timezone info in the Date header (Steffen Goeldner).
So, create a message using its local CRLF, but:
$CRLF= "\x0a\x0d";
sub _add_to_header {
my ($class, $header, $key, $value) = @_;
return unless $value;
${$header} .= join(": ", $key, $value) . $CRLF;
}
A string is built up with _add_to_header, then passed to Email::Simple->new.
Since it will end with a single $CRLF, the doubled-crlf detector will not
determine that \x0a\x0d is the crlf in question, and will, instead, use "\n"
The header, though, will actually be line-ended with CRLF. Email::Simple can
parse this properly, but it seems Wrong. It also has nothing to do with what
ends up in the body, which is generally passed in as a string with its own,
uninspected linefeeds.
I'm going to do some testing, but I think the right thing to do is to try to
generate a header with CRLF dividing the headers and dividing the head from the
body. I am not excited at the prospect of re-line-breaking the body, though.
Beyond vague feelings of The Right Thing, is there a strong practical reason
for any particular behavior?
--
rjbs
Mail::Audit: dismembered, but not dead
I use Mail::Audit at work, for historical reasons. To try and reduce the number of parts that are contained in the core distribution, I've split the PGP, Razor, and ListDetector plugins into their own dists as of today. I may split out the MAPS plugin and maybe a few others soon. I don't think this will affect anyone: existing Mail::Audit scripts will continue to work normally, and I don't think many new scripts get written these days using Mail::Audit. -- rjbs
Email-ARF (abuse reporting format)
AOL lets you say, "Look, I want to be a nice guy. If you get complaints about mail from me, please let me know and I'll look into it." They send these report in a format called ARF. http://www.mipassoc.org/arf/ Well, actually, they use two formats, but one is old and lame and is being replaced by ARF. There exists an ARF-producing Perl module, MIME::ARF, but it's not on the CPAN and only produces reports. Today I wrote an ARF-reading module, Email::ARF::Report, and uploaded it. It's a prototype, and if you deal with ARF, please let me know if you think it's ready to go to production. Here is the announcement I sent to the ARF list: http://mipassoc.org/pipermail/abuse-feedback-report/2007q1/000109.html I may or may not produce an Email::FeedbackReport::AOL to parse their old report format. It's pretty lousy, so I'm not sure it's worth it; there isn't much information to glean from it. -- rjbs
Re: Email-ARF (abuse reporting format)
* Ricardo SIGNES <[EMAIL PROTECTED]> [2007-03-20T22:39:49] > There exists an ARF-producing Perl module, MIME::ARF, but it's not on the CPAN > and only produces reports. Today I wrote an ARF-reading module, > Email::ARF::Report, and uploaded it. It's a prototype, and if you deal with > ARF, please let me know if you think it's ready to go to production. Today, I uploaded 0.001, which has a method to create a new report. The method isn't so bad, but needs some redesign. I don't like that "new" returns a report and "create" returns a MIME message. (I'm not a huge fan of the new/create dichotomy, but we've got it!) -- rjbs
Re: Mail Help Require
* Hans Dieter Pearcey <[EMAIL PROTECTED]> [2007-04-19T12:08:54] > On Thu, Apr 19, 2007 at 10:42:46AM -0500, Karen Cravens wrote: > > If it's direct filesystem access, you can look at Email::Folder for other > > access methods. If it's IMAP, I don't know that PEP is the route to go. > > IMAP definitely gets beyond Simple - maybe someone will jump up to correct > > me, but I don't think PEP handles IMAP. > > http://search.cpan.org/dist/Net-IMAP-Simple/lib/Net/IMAP/Simple.pm > > is the module I see most commonly recommended. Right. There exists Email::Folder::IMAP and ::IMAPS, but they're quite limited, list the rest of Email::Folder. -- rjbs
Re: Email-Filtering, what is used nowadays? (and integration w/ SpamAssassin)
* Joachim Schrod <[EMAIL PROTECTED]> [2007-05-07T13:09:49] > The description page for Email::Filter itself is sparse, to say the > least. For Mail::Audit, at least some future plans are written down; > no such information about Email::Filter. (E.g., what about missing > functionality for migration from Mail::Audit, or a substitute for > Mail::Audit's procmail migration support). The situation is something like this: * I am a big loser, and still haven't quite converted my own mail filtering from procmail to Perl. (Instead, I've written a lot of Perl to generate my procmail configuration.) * Where I work, we use some long-established code that uses Mail::Audit. This means that I have a pretty strong incentive to work on Mail::Audit, and less to work on Email::Filter. This is too bad, because Mail::Audit is definitely the crazier code. > As a further example, let's take this open bug ticket: > http://rt.cpan.org/Public/Bug/Display.html?id=25027 > Does this really mean that it is hard to migrate the logging > component of one's Mail::Audit setup? Yes, probably. Email::Filter is really very minimal. It doesn't have logging built in, so it would need to be added by a plugin. Unfortunately, the existing triggers are not well-documented, and they're very minimal and could use enhancement. (I have often found that the Email:: code is nice and simple, but usually just a little too simple for me to /use/; I try to add the needed stuff without compromising the simplicity.) For that logging bug, there's an ideal place to add a trigger, but there's no way to get the needed data. > But anyhow, as I wrote: I have a working setup with Mail::Audit, and > that module is listed in PEPMaintained, so I thought it is a > maintained PEP module. Nevertheless I consider a switch, if that is > considered best-practice and if somebody could please explain why it > is best practice, could point out some advantages of the new module. If I were you, I would stick with what works. If you have some tuits, it would be great if you wanted to help make Email::Filter more pluggable, and then port yourself. I just don't think you need to switch if it isn't broken for ou. > Thus I'll repeat my questions: > -- Is Email::Filter faster? Probably. > -- Maybe its code quality is better: > -- Has it fewer known errors? I don't think so, but only because you said "known." The M::A code is a mess. The Email::Filter code is very simple. > -- Or is it bug history better, fewer bugs over time? > -- Maybe it is known to abort more seldom? > -- Or is it otherwise more stable? How? Email::Filter seems (based on my gut) so much less used that it's hard to say. Its bugs tend to be, "why can't I easily X?" while Mail::Audit's bugs tend to be "why does X fail in this weird way under these circumstances?" > -- Is its memory footprint smaller? Almost certainly. > -- Are there plans about future features that will be easier to > uptake if I switch now? I have no plans for it, specifically. Yet. > -- In particular, better SpamAssassin integration than the two > lines of obvious code from Email::Filter::SpamAssassin? In my own prototype of an Email::Filter script for myself, I used my Mail::SpamAssassin::SimpleClient. I agree that SA needs to be way, way easier to use from E::F. > One or two (or more :-) of such advantages would be a very good > incentive for the additional work that needs to be spent migrating a > working installation. Maybe, not only for me; this could be something > for the PEP Wiki. (And if some folks would answer these questions, I'm > willing to spend the time to add a summary of the answers to the > Wiki.) Yeah, I need to get better at adding "to do" items to the wiki, so that volunteers or knowledge-seekers can more easily see what's done and what isn't. -- rjbs
Re: Email::MSG
* Matijs van Zuijlen <[EMAIL PROTECTED]> [2007-05-11T09:25:57] > I am the author of msgconvert.pl (http://www.matijs.net/software/msgconv/), > a tool to convert Microsoft Outlook's .msg files into message/rfc822. I'm > in the process of making this tool into a module. Excellent. > From the current contents for the Email:: namespace, I conclude that the > name Email::MSG would be a good name for this module. The module will be > able to parse a .msg file, and produce an Email::MIME object from that: > > my $msg = new Email::MSG $filename; > my $email_mime = $msg->to_email_mime; I agree with HDP: It sounds too much like "Email::Message" and thus a replacement for Email::Simple. I'd go with Email::MIME::FromMSG or something on those lines. What can you do with an object of your class besides to_email_mime? > - When making a multipart message, Email::MIME inserts Date and > MIME-Version headers into the parts. IMHO, these shouldn't be there. Isn't that annoying? We've talked about seamless ways to fix this, but it isn't done yet. > - In a multipart/alternative message, there usually is a text like 'This is > a multipart message in MIME format' outside the actual alternatives. > Email::MIME does not allow such a text to be added. Right. I will probably add the ability to set the prologue and epilogue text at aome point. It will be nice to at least have the usual prologue in place. (Patches welcome.) -- rjbs
Re: Announce: Email::Folder::Exchange
* Smith Warren - wasmit <[EMAIL PROTECTED]> [2007-05-14T09:44:01] > I've just uploaded Email::Folder::Exchange to CPAN to allow > Email::Folder access to Microsoft Exchange folders via WebDAV (Outlook > Web Access). Simply provide your OWA Inbox url, username, and password, > and you can browse your mailbox. Hey, I'm sorry I didn't reply to this immediately. I saw it appear in the CPAN uploads queue, and I said, "Cool!" I could've used this five years ago... if OWA had existed in any useful form then. > I would also propose the following minor additions to the Email::Folder > API, since many email folder providers support a hierarchy of folders: Well... wait for my next email. -- rjbs
99 email problems (but a bounce ain't one)
I've been meaning to update the PEP Wiki with more useful information, mostly in the form of signposts to would-be contributors or module evaluators. (I'd like to add more information and how-to articles, but I'm running a little short on time, lately.) I added problems and ideas sections to a number of pages, this morning. You can always see what pages have problems or ideas listed, here: http://emailproject.perl.org/wiki/Category:HasProblems http://emailproject.perl.org/wiki/Category:HasIdeas I think the three most important modules to fix, or at least establish an ideal plan for, are: Email::Simple, Email::Send, and Email::Folder. Email::Simple, here, encompasses most of its many subclasses. It needs to be easier to write plugins, and they need to work in some way that is not just "cram stuff into other packages." http://emailproject.perl.org/wiki/PEP_Plugin_System I've written about Email::Send's problems, before, and I won't go on about them, again, here. I will seriously get Email::Sender committed, such as it is, soon, for comment. Email::Folder was my main point of thinking, this morning. The name promises to be a center of functionality for all things related to managing a collection of messages. Instead, you get a reader, and other functions rely on the "folder represented as a string" idiom of Email::FolderType. Does anybody else want to say anything about this before I start blathering? -- rjbs
Re: Email::Send cram-md5 Authentication failure
* [EMAIL PROTECTED] [2007-05-23T01:03:14]
>
> sub SendEmail {
>
> my $MessageRef = shift @_;
> my $SMTPHOST = shift @_;
> my $Sender = Email::Send->new({mailer => 'SMTP'});
>
> if ($SMTPHOST eq '') {
> $SMTPHOST = 'localhost';
>}
> $Sender->mailer_args([Host => $SMTPHOST, Debug => 1, Hello =>
> 'radacted',
> username => '[EMAIL PROTECTED], password =>
> 'blahblah']);
>
> $Sender->send(${$MessageRef});
>
> }
Can we safely assume that your actual code is NOT missing a quote at the end of
the username string?
Email/Send/SMTP.pm does show that username and password are used as the args.
> Net::SMTP=GLOB(0x1927168)<<< 220 server.domain.blah ESMTP
> Net::SMTP=GLOB(0x1927168)>>> EHLO mail.domain2.blah
> Net::SMTP=GLOB(0x1927168)<<< 250-hostname.domain.bblah
> Net::SMTP=GLOB(0x1927168)<<< 250-AUTH LOGIN CRAM-MD5 PLAIN
> Net::SMTP=GLOB(0x1927168)<<< 250-AUTH=LOGIN CRAM-MD5 PLAIN
> Net::SMTP=GLOB(0x1927168)<<< 250-STARTTLS
> Net::SMTP=GLOB(0x1927168)<<< 250-PIPELINING
> Net::SMTP=GLOB(0x1927168)<<< 250 8BITMIME
> Net::SMTP=GLOB(0x1927168)>>> AUTH CRAM-MD5
> Net::SMTP=GLOB(0x1927168)<<< 334 cram challenge text here
> Net::SMTP=GLOB(0x1927168)>>>
> XNwYW5uQHdlYnNvZnR0dC5uZXQgYzViNjhjMTUMDUzNjg3MmYwMmU3ZGQ=
>
> Net::SMTP=GLOB(0x1927168)<<< 535 authorization failed (#5.7.0)
I don't know what to tell you. I'd suggest trying to reduce this to a problem
with Net::SMTP, and I suspect that you will find it also fails there.
I suspect that this is really just a username/password problem, but you'd know
better than I.
--
rjbs
Email::LocalDelivery streaming interface
The last bug of the day was, "Why is this process getting so big?" See, it was
getting an HTTP upload, creating an Email::Simple::FromHandle, then delivering
it. It should all have been streaming around, nothing in memory.
Then we realized that Email::LocalDelivery was dealing with the string at once,
because it was written before ::FromHandle. Here's the bizarre thing, though:
Email::LocalDelivery accepts only a string to deliver. In fact:
croak "Mail argument to deliver should just be a plain string"
if ref $mail;
It calls the deliver method on the various Email::LocalDelivery::* classes,
like ::Maildir, which says:
sub deliver {
my ($class, $mail, @files) = @_;
$mail = Email::Simple->new($mail)
unless ref $mail eq "Email::Simple"; # For when we recurse
Unfortunately, ::Mbox does nothin of the sort, only writing a string to a file
with no provision for receiving an Email::Simple object, so the exception in
Email::LocalDelivery can't really be fixed without bizarre consequences:
partial deliveries based on the delivery targets.
I'll probably fix this in all the E::LD:: classes I control, then try to get
the other CPAN modules fixed, then hope nothing on the darkpan breaks. At
least I can fix the streaming delivery in Email::LocalDelivery::Maildir and
call its deliver method directly, because hey, THAT's a great idea...
Thoughts?
--
rjbs
Re: YAPC-NA
* Karen Cravens <[EMAIL PROTECTED]> [2007-05-25T22:14:39] > So I'd remarked to my husband (a sysadmin) that I'd really like to go to > YAPC in Houston, and he mentioned it to his boss, and one thing led to > another, and hey, now they're sending *him*... paying mileage, hotel, > conference fee, and meals. And being that he's driving, I don't even have > to hide in a suitcase to go along for free. So aside from my meals and my > conference fee, Hustler's gonna pay for everything. How cool is *that*? That's the life. I just hope you don't have to be driven too far. More than six hours in the car and I start to lose my will to live. > So, on to the important question: What style of beer ought we be bringing? If I could bring beer (we're flying, and I refuse to check bags), I would bring something from Victory, I think. M. Hop Devil! As for what you should bring? Something good, of course! In my book, that could be anything but stout. -- rjbs
toward Email::Simple 2.000
First: In case you're subscribed, note that the pep-checkins is currently busted. I haven't even looked into it, yet. We moved PEP svn from a standalone machine to a Solaris zone recently, and I probably broke something stupid. I bring this up because I know that it means none of you have probably seen the Email::Simple checkins I made yesterday. I'm trying to get ready to release Email::Simple 2.000. It's going to be a great release, I think. I'm not trying to do the crazy "not-backwards-compatible changes at major version number!" thing. Instead, I'm trying to document and standardize more of the interface. My mantra for the 2.x series of Email::Simple is "better interface standardization." I want it to do roughly the same amount of stuff, with roughly the same performance. I just want it to be clearer how one can subclass Email::Simple, and I want to refactor, document, and improve the usability of the core features of Email::Simple. (This will probably include a rewrite of the Email::Simple<->Data::Message relationship.) Here's a summary of things I checked in last night: * better documentation for the Email::Simple::Header object * provide a means to request a different header class * public interface for header folding * documented how options are passed to Email::Simple constructor * add options to as_string, namely options to alter header folding * add methods that return default values of various options * some performance improvements These changes are mostly here to make it easier to subclass Email::Simple's behavior. In almost every case, I want the answer to "how do I change Email::Simple's" behavior to be "subclass" or (later) "use a plugin." Since some features are really very fundamental core features, though, they seemed like a good place to demonstrate my intentions. I'm going to release this as a _x release, soon. Please let me know, now or then, if you have any thoughts. -- rjbs
Re: Email::MSG
* Matijs van Zuijlen <[EMAIL PROTECTED]> [2007-06-15T13:53:39] > Nothing much at the moment, but I might add the ability to directly access > its properties rather than always having to convert to an Email::MIME > object. > > Maybe Email::Outlook::Message, leaving open the option of adding > Email::Outlook::Archive in the future, for accessing Outlook's PST files? That sounds good. -- rjbs
Email::Sender in Subversion
It's been festering in my ~/tmp for too long, half ported from Work Code, so I've put it in Subversion, hoping that either I'll feel more compelled to get it release-worthy or someone else will. Heck, maybe someone will just have some snide remarks. It would be nice to get this more demonstrable by YAPC. -- rjbs
notes from YAPC
YAPC is done. I gave a talk on handling email with Perl, and I think it went fairly well. We had a get-together after the talk and spoke about what we need to do in the future. I think that also went well. I'm going to start dumping out some of the results from all of that over the next few days, and I hope we can start building some momentum. Here's a dump from my OmniFocus for PEP tasks. Some of them are stupid and only for my notes, some may provide some reminders to people who were (or weren't) there about things we'd like to start talking about or working on. Anyway, I'm on a bus and I think I'd like to try to get a few more disconnected tasks done before I arrive at the office. More soon! build db of known email headers fix all copyright notices standardize dist layouts add MinimumVersion tests to * Email-Simple-Header finalize api for 2.0 Email-Corpus design api basic implementation plan Mail-LocalDelivery actually use it in Mail::Audit (or E::LD or E::Sr) improve email sending tested Email-Sender Mail::Message::Coerce::Email colon / semi typo in Mail::Mailer docs Email::Base BounceParser 2.0 Email::Metadata Email::KB Email::Return (Zion) -- rjbs
Email::Simple 2.0 immanent
I've re-privatized some of the extra API I had added to ::Header in order to get a release out while still waffling a bit about how much interface to add to Email-Simple itself. The current trunk of Email-Simple is, I think, ready to be released as Email::Simple 2.000. This is not some magic 2.000 all new API or anything. It's just the version after 1.999. Please have a look, give it a go, and let me know if there's any reason to hesitate. I plan to release this week. -- rjbs
Re: Email::Simple 2.0 immanent
* Ricardo SIGNES <[EMAIL PROTECTED]> [2007-07-03T18:09:01] > I've re-privatized some of the extra API I had added to ::Header in order to > get a release out while still waffling a bit about how much interface to add > to Email-Simple itself. Email::Simple 2.000 has been released. Also, 2.001. Also, 2.002. We're at 2.002 and holding. Honestly, pending other bugs, I see very little reason for Email::Simple to change further in the near future. We've got much more pressing work in the other projects that we've talked about getting under way. More on those, soon. -- rjbs
simpler than simple, the newest e-mail module
For those of you who read my journal, I apologize for the duplication. For the rest of you, I apologize for the content. http://use.perl.org/~rjbs/journal/33806 -- rjbs
things you can do
I've gotten a few "What can *I* do?" emails lately, so here's a recap. These are all things that should be do-able without having to learn anything really esoteric, make any design decisions, or worry about breaking things too much: Dists desperately in need of more tests (roughly in order of urgency): * Email-MIME, Email-MIME-Modifier, Email-MIME-Creator * Email-Address - there are some cases in which it still seems to hang * Email-Filter * Email-MIME-Attachment-Stripper * Email-LocalDelivery - it used to fail to report some class of errors * Email-Sender * Email-Simple, Email-Simple-Creator * Email-Thread * Mail-Audit Determine minimum version: * all dists should get a t/perl-minver.t like Email-Simple has * the minimum version Perl::MinimumVersion suggests should be documented * ...and tested to ensure that it's right * ...and corrected if it is not Wikimastering: * just dump ideas and other thoughts onto the wiki! Miscellaneous other tasks: * port any changes to the Mail::Audit delivery routines to Mail::LocalD * ...and then make Mail::Audit use that * update Email::Valid to use Email::Address for RFC2822 address checks Then, here are some more technically-involved tasks: * the Email::Abstract enhancements listed here: http://emailproject.perl.org/wiki/Email::Abstract * improve Email::Address to require no preprocessing of whitespace * improve Email::Address to handle address groups (at least the empty one) * make Email::MessageID not isa Email::Address * make Email::MessageID use Sys::Hostname::Long if available -- rjbs
pep-checkins is operational again
About three months ago, we moved pep.pobox.com (which hosts emailproject.pobox.com) to a new server. A few things broke for a few hours, but svnnotify hasn't been working since then. I "fixed" it a few times, but never bothered checking that it was fixed, since I viewed the commit list as unimportant. Since I want people to be able to see WTF is going on, though, I've realized that the commit list is actually very important. It's been fixed as of this morning. To join pep-checkins, visit http://listbox.com/subscribe/?list_id=4142 -- rjbs
replacing the BounceParser
I'm not sure how many of you have used "the BounceParser." We use it
extensively at work, and speak of it only in hushed tones -- or, if we're sure
it can't hear us, we proclaim its wickedness loudly, dreaming of a day when we
can be free of it. In the meantime, though, we suffer through its horrible
reign of terror.
Mail::DeliveryStatus::BounceParser sucks. If you don't already know this, here
is a primer on what to hate:
* MDSBP->new($input) either returns a BounceParser object or false
* the BounceParser object represents a bounce, not a parser
* and that's why you can call it as ->parse, now, too
* hey, it's 400 lines long, so maybe it deserves two names
* if it returns false, you still might not have a bounce
* falsity means "we're sure it's not a bounce"
* an object means "it might be a bounce, but check ->is_bounce"
* a true result from is_bounce means we're sure it IS a bounce
* an ideal bounce is an RFC1892 message
* other messages are s///d to look more like RFC1892 before parsing
* a bounce has many reports
* you can get report data with, for example, $bounce->addresses
* but don't try $report->addresses, that doesn't work
* reports have std_reason fields, explaining WHY the message bounced
* one of these is standard reasons is "no_problemo"
* not one of them is "spam" or "blacklisted"
* go look at the source
After all that, it just doesn't work very well, either.
I want a good bounce parser for two main reasons:
1. Recognize bouncing mailing list subscribers so they can be booted.
2. Recognize bounces that say that my MXes are being blocked so I can
address the situation.
I'm not sure what this system needs to look like, but from 1,000 feet up, the
BounceParser isn't totally wrong.
my $bounce = $parser->parse($email_abstract);
my $remote = $bounce->bounced_by;
my $local = $bounce->originally_sent_by;
my @reports = $bounce->reports; # may be ()
my $reason = $report->bounce_reason; # one of a fixed list
my @addrs = $report->bounced_to;
I don't see any reason for this to notice autoresponders at all. If we can
tell they're not bounces, there's no reason to report them, is there?
The parse method probably does something like:
for my $plugin (@prioritized_list) {
return $plugin->parse($message) if $plugin->can_parse($message);
}
return;
Have I been twisted and blinded by The BounceParser? Is there a significantly
better way to design this?
I have a few million bounce messages sitting in an archive, just waiting to
become test cases.
--
rjbs
Email::Base (ad Email::Sender)
I am eager to get Email::Sender ready for use. The prototype that I use at
work is one of the most useful pieces of code in our internal toolkit, I think.
It makes all sorts of email-related testing much simpler.
Toward that, I've been doing some of the simple stuff required to get
Email-Sender in shape, and now that much of the simple stuff is done, I'm going
to start writing (stealing) more tests. That's not enough, though. For those
of you who saw my talk on PEP at YAPC::NA, you know that one of my biggest
dislikes about Email:: is the way that its modules *utterly* fail to display
signs of design for organic interoperation. (Today's amusing nit:
Email::Abstract doesn't use Email::Simple's method names for common
operations.)
When we spoke about solving this problem at YAPC, we talked about a small set
of common behaviors for new Email:: modules. These would form Email::Base, the
name of which has been used to mock the idea of Email:: being overly complex.
To that, I say, "whatever." We, the current custodians of that code, don't
want to turn it into a giant monolith. In fact, I think it's safe to say that
Email::Simple won't take on the Base behavior at all, both to stay as small as
it is and because it would require too much reworking.
That said, I think Email::Base is a good idea. It can certainly be seen as
step one in the PEP Plugin System, which should help make things easier to deal
with.
At YAPC, the basic idea was that Base would need to include two kinds of
things: exceptions and attributes.
I think exceptions are simple: Exception::Class is very popular, very good,
and fairly lightweight. It works, I believe, on 5.005. Its prereqs are
Scalar::Util and Devel::StackTrace (both of which we'd probably need anyway)
and Class::Data::Inheritable, which doesn't strike me as a big worry.
Possibly the cheapest option for attributes is Class::Accessor, but I think
it's too simple. I think it's important that we have a system that makes you
really notice when you're violating encapsulation. A lot of Email:: bugs have
come about because of a cavalier attitude toward doing things like
$email->{head}{names} = $whatever. Once that's in Email::Foo, you can't change
the internals of Email::Simple without trauma.
I'm not sure what we really want, and I am eager to hear ideas. I would like
to keep things fairly simple. Here's a simple idea whose merits and flaws I
have not deeply considered, yet (it just popped in there):
Objects are hashes in the form { namespace => $guts, ... }. Guts are
namespace specific. The objects' classes also have a mapping of attributes
to providing namespaces. When you say $obj->attr('foo') it does something to
get it from the namespace that owns 'foo'. When two plugins attempt to claim
authority over the same attribute, an exception is raised, at compile time if
possible.
Ok, I know this was a little wordy, but I'm eager to get this ball rolling, so
I'm just going to send it. Replies anticipated eagerly.
--
rjbs
Re: Email::Base (ad Email::Sender)
* Dave O'Neill <[EMAIL PROTECTED]> [2007-07-18T14:10:54] > > I'm not sure what we really want, and I am eager to hear ideas. I would > > like to keep things fairly simple. Here's a simple idea whose merits and > > flaws I have not deeply considered, yet (it just popped in there): > > [ ... ] > > My random thoughts: > > I'm not too keen on introducing Yet Another Way of doing objects. Do you > think name collisions are going to be a huge problem? Is trying to > enforce namespaces for attributes really going to buy us anything, when > someone could just implement an Email::Whatever that extends Email::Base > and mucks about in the top level of the hash for its object data anyway? Name collisions are not a problem until they are, at which point they are a massive problem, because some schmendrick calls his autoreply plugin's method "reply" which someone else is using for "is_reply?". That said, the real solution may be to look for the redefinition warnings and pay attention to wtf you are doing. I am definitely with you: I am not excited about introducing another way to do objects if we can happily avoid it. > What about inside-out objects? It might be handy for dealing with > hundreds of lightweight objects (headers, message metadata for all > messages in an mbox, etc). I have no a priori objection to them. Is there a system for building them that is fairly lightweight and portable? We should try to establish some rough guidelines about what the goals of PEP-produced code *are*. I think Email::Simple needs to keep having roughly zero pre-reqs and keep running on 5.005. I am probably fine with Email::Base requiring 5.6 and having a few very useful prereqs. I'm not sure how many "a few" or how useful "very useful" are. I mean, do we want to use Moose? It always comes up, half in jest. I don't know what its performance or maintenance impact are. I don't think it adds *that* many prerequisites, but I haven't looked up the whole tree. > And... another random thought on objects. Do we want to enforce a > naming convention for accessors? YES. So much. > get_foo() and set_foo() vs a foo() method that changes behaviour based on > args? I like the get/set version mostly because it's easier to grep for in > the code, and harder for someone to accidentally screw up. I don't mind get/set. I also like mutators. I think I'd prefer get/set. More importantly, I want them to be freaking standardized. I am so tired of: $frob->set_foo( $thing->foo); $object->foo_set( $thing->foo ); -- rjbs
Re: Email::Base (ad Email::Sender)
* Smith Warren - wasmit <[EMAIL PROTECTED]> [2007-07-18T16:09:21] > Just remember, if you ever want your module to become part of the perl > core distribution, it must rely solely upon other core modules. I don't. Email modules do not belong in the core, imho. > Secondly for those of us who have brain-dead admins, installing modules > is quite the chore. I would rather install one module than many. It drives me crazy that people say this -- although I do believe it's true -- beacuse in almost all cases it could be "cpan Email::Whatever" or even "apt-get install libmail-whatever-perl" > All of the syntatic sugar that Moose [and any other meta-oo class] > affords can be implemented with common (although seemingly complicated) > standard perl structures. That's true, but it's a lot like saying, "Anything that Perl does can be done in assembly, although it may take more lines." -- rjbs
Re: Email::Base (ad Email::Sender)
* Hans Dieter Pearcey <[EMAIL PROTECTED]> [2007-07-18T16:04:17] > On Wed, Jul 18, 2007 at 03:52:55PM -0400, Ricardo SIGNES wrote: > > I mean, do we want to use Moose? It always comes up, half in jest. I don't > > know what its performance or maintenance impact are. I don't think it adds > > *that* many prerequisites, but I haven't looked up the whole tree. > > Specifically, I keep bringing it up, because there are so many possible > different behaviors that you might want to mix together, especially once MIME > is involved, and Moose seems like a really solid framework for managing all of > that without a ridiculous number of subclasses (I'm thinking of Roles, in > particular). Yeah, I think that any useful solution is going to be a form of roles, mixins, traits, or whatever you want to call those things. I just don't know whether it's Moose's implementation that will win out. I'm trying not to prototype any of this in my head in terms of email objects, because the minimum useful core is very large. I am fairly convinced that a core email object on which other behavior is hung will be MUCH more like Mail::Message than Email::Simple. Still, if the common code is going to support something on that order, then scalability is as important as lightness. -- rjbs
Email::Abstract: big updates; test please!
So, I spent a few hours today writing more comprehensive tests for Email::Abstract. It started when Mark Overmeer noted that he thought there was a bug in the way Mail::Message objects' newlines were handled. I had a look at the tests for Email::Abstract and basically found that there were a huge number of bugs not exposed by the lousy testing. I've tried to fix them all, and I've gotten 100% test coverage, with much more assertive tests. Here are some of the most important bugs I've addressed: * newlines were lost in the bodies of Mail::Message and Mail::Internet objs * headers were not unfolded from Mail::Internet and MIME::Entity objects * the means of determining the adapter classes were not consistent I'm a little worried about some of the changes, especially my forcible unfolding of headers for Mail::Header objects. Then again, I believe things are correct NOW and it's hard to imagine anyone relied on this BEFORE. I want to use Email::Abstract in Email::Sender, so getting this right is important. I want to make sure the existing behavior is good before I add anything else -- and I will definitely add a stream_to (print_to?) method in a release soon. So, testing, patches, and / or review appreciated. It's in svn and on the CPAN. -- rjbs
Re: replacing the BounceParser
* Dave O'Neill <[EMAIL PROTECTED]> [2007-07-18T15:30:21]
> Maybe the solution is just to not call it BounceParser?
Yeah, I'd had this thought myself, and I have no idea why I ever rejected it.
Maybe I was afraid of being too broad. Maybe I just got jaded.
I think you are absolutely right.
> # Create a parser. Don't load any plugins automatically.
> my $parser = Email::Classifier->new();
...but allow them to be specified in the constructor, surely.
> # Load the Email::Classifier::Plugin::Bounce plugin for catching
> # bounce messages
> $parser->add_classifier('Bounce');
...where this really means something like "create an object from the
ECP::Bounce class with the defaults."
We talked on IRC about a few things worth repeating:
Rather than $parser, I think it should be a classifier. Classifiers use
classifiers to classify, making it easy to implement the Bounce classifier in
terms of more specific bounce classifiers.
We want to be able to give arguments for the classifiers:
$classifier->add_classifier(GPG => { secret => 'special.gpg'});
...and that mean we should be able to specify the plugin more than once,
meaning that we should be able to give them identifiers:
$classifier->add_classifier(SpecialPGP => GPG => { secret => 'special.gpg'});
$classifier->add_classifier(DefaultPGP => 'GPG');
And if we got no identifier (or maybe if we did) we can identify them by stack
trace:
Error while processing classifier SpecialPGP...
or
Error while processing classifier added at foo.pl, line 28
I like that technique so much that I think I'm going to steal it from myself.
> # Iterate over prioritized plugins and return first hit
> my $result = $parser->parse( $email_abstract );
I wonder what we want to pass in. It needs to do multipart, because even for
the original case of bounces, we may have attached reports, etc. The idea of
Email::Abstract doing multipart sort of scares me, but it might be alright if
it's really toned down -- you use a subclass or mixin that automatically
replaces uses of Email::Simple with ::MIME, and then each subpart is another
Email::Abstract::Multipart. I don't know.
> my $r = $parser->parse( $email_abstract );
> if( $r->is_gpg_signed() ) { # let plugins provide new methods to
> # result object (inheritance? Sub::Exporter?)
> if( $r->valid_signature_by('0x12345678') ) {
> # Yes, we fully trust the message now
> eval $email_abstract->get_body();
That is pretty evil!
Unrelated to the ev[ia]l, I wonder if there's a compelling reason to use
->is_gpg_signed instead of ->get_attr(...) -- but then I think that the "method
missing" exception when you typo is probably compelling enough!
Anyway, as I said before, I will say again: I think you're spot on.
--
rjbs
more Mail::Box interop
Awesome! I mentioned, at YAPC, wanting to get more interoperability between Mail::Box and Email::*. As of this morning's release of Mail-Box, there are native conversions from Email::Simple and Email::Abstract, and there is a native conversion to Email::Simple. They're crude now, but they'll get better. This should make it easier to delay loading Mail::Box until needed, for example. -- rjbs
Email::Base 0.000
Everyone following along with the svn checkins? If so, this won't be news. I've added a super-rough first draft at Email::Base. All it does is require Email::Exception and provide a ->throw method. It works like this: $object->throw($exception_moniker => $arg); The moniker is either '::Foo' to get Email::Exception::Foo or 'Bar::Baz' to get just that. I don't think I like this, but it's where I started. The exact opposite might be better. $arg is either a string, which is used as the error param to the exception, or a hashref, which is used as the entire set of params. A "thrower" param is always added, referring to the object on which throw was called. Email::Exception has only one field in the base class: thrower. Next email: Email::Sender. -- rjbs
Email::Sender updated to use Email::Base
So, now Email::Sender isa Email::Base, and uses ->throw.
I'm not 100% comfortable with the current Sender behavior. I've changed it a
few times, but here's the rough outline: a sending with a sender seems to have
three possible outcomes:
1. everything worked!
2. it worked, but not for every recipient or destination
3. it didn't work at all
These are currently handled like this:
1. return true with "return $self->success"
2. throw an Email::Exception::Sender::PartialFailure
there's a ->failures field that's a hashref, { $dest => $reason }
2. throw an Email::Exception::Sender::FailureFailure
That means that you must always eval the ->send call and check for both partial
and total failure, if you want to be handle all three cases.
So, first of all: any immediate and personal reactions?
Here are some of mine:
How many people are going to shoot themselves in the foot by re-sending to all
recipients after a PartialFailure? How many would shoot themselves in the foot
by not noticing failed destinations if partial failure was treated as a form of
success? I think the current behavior is righter, and no likelier to trick
users into messing up.
Do we need to have a built-in distinction between temporary and permanent
failure?
What are the $reason data? Strings? If so, how do we represent a mixture of
temporary and permanent failures? (Is this possible? I am not going to check
the RFCs at this second, but during SMTP, can the remote respond with 4xx's
to some RCPT commands and 5xx's to others? I don't see why not.) Exceptions
may make suitable values. It makes a multiplexing Sender easy to write,
anyway.
--
rjbs
Re: Email::Sender updated to use Email::Base
* Dave O'Neill <[EMAIL PROTECTED]> [2007-07-24T07:41:50]
>
> Maybe ->failures() could return:
>
> [
> {
> address => '[EMAIL PROTECTED]',
> type=> 'temporary',
> message => '451 4.3.2 Please try again later'
> },
> {
> address => '[EMAIL PROTECTED]',
> type=> 'permanent',
> message => '550 5.5.1 User unknown',
> }
> ]
>
> Reasons for this:
>- hashrefs in the list can be upgraded to objects at some point in
> the future if necessary, and still retain some backwards
> compatibility.
In that future, people will be saying ->{type}. I wonder if it isn't simpler
just to start with a minimal object so that everyone is always using methods to
begin with.
>- The address/type/message are probably the lowest common denominator
> for all sender backends. We can fill the hash with extra data,
> should the sender have it available (SMTP code parsed out nicely
> for us, or return code from commandline invocation, etc).
Do we want to include a reference to the sender which finally tried and failed
to send?
--
rjbs
Re: Email::Base 0.000
* Hans Dieter Pearcey <[EMAIL PROTECTED]> [2007-07-24T22:00:15]
> On Tue, Jul 24, 2007 at 07:46:36AM -0400, Dave O'Neill wrote:
> > Using '::Bar::Baz' is probably a better choice. Most Perl people know
> > what a :: prefix means for namespacing, but only Catalyst people will
> > find '+Bar::Baz' intuitive.
>
> I think this is worse, honestly; I wrote a whole reply to you assuming that
> you meant ::Bar::Baz would have Email::Exception prepended, because that is
> how I instinctively parsed it.
->throw('Fully::Qualified::Name');
->throw('-Under::EE');
(a) hypen means "tacked on to previous thing" like "hyper -active and -trophic"
(b) it lets you skip quoting
(c) which means less typing for the common case
->throw(-Under::EE => { });
I dunno, it's late and I'm about to sleep, but it's a thought.
--
rjbs
base exceptions
Right now, Email::Sender uses -Sender::Failure, and two subclasses of that. The only core exception (beyond the root class) that I can imagine is Unimplemented. Thoughts? -- rjbs
MIME-Lite in pep svn
As those of you on pep-checkins saw (sorry for the huge noise), MIME-Lite is now in the PEP subversion repository. I've made a new dev release with zero code changes. I'm not a fan of MIME-Lite, as it's fairly buggy and not all that lite. I'm not really interested in making it the best MIME generator around (though feel free to do that), but I would like to reduce its bugginess, since a lot of things use it. As those of you on pep-bugs know (sorry for the brief outage; my bug aggregator script is lame), there are about 120 bugs against email-related modules. 31 of those (that is: 25%) are filed against MIME::Lite. Testing so that 3.020 can be released would be great. Commits to fix bugs would be great. Be great! -- rjbs
Re: Messages won't import into MySQL
* Chris Miller <[EMAIL PROTECTED]> [2007-08-03T19:26:08] > Most messages import just fine, but I'm running into more and more messages > that don't make it. Sendmail complains with the following : > > dsn=5.3.0, stat=unknown mailer error 255 > > The message content gets inserted into the 'mail' table, but the message ID > never makes it into the 'mail_date' table. Unfortunately I use the mail_date > table to sort messages by date. I can have a brief look, but I have never, ever used Email::Store. I only updated a few CLEARLY busted things to bring it under the care of PEP. I think there are some others here who have more experience with it. -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Christopher Nehren <[EMAIL PROTECTED]> [2007-09-14T17:00:51] > We've got a (mostly empty for the moment) repo at > http://www.coitusmentis.info/repos/email-archive/ , hosted on my home cable Would you like to host this on emailproject.perl.org, on a nice Sun server at a datacenter? -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Simon Wistow <[EMAIL PROTECTED]> [2007-09-14T18:39:33] > As for Perl MLMs - have you looked at Siesta? It's a bit rough and ready > in some places but I've been running mailing lists off it for years. On the topic of Perl MLMs, I work at Listbox.com, something of an "enterprise" MLM. I'd like to help get what bits of its code can be made useful for reuse out into the world at some point, so anyone working on this stuff should feel encouraged to bug me (on the list) for thoughts, patches, or code. -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Karen Cravens <[EMAIL PROTECTED]> [2007-09-19T11:53:13] > sinus yuck and can focus on finishing the install, we can move over there. > (Guess I could just turn on the daemon and hope for the best...) Of > course, once I get off the Sudafed+Dew I probably won't be so inclined > toward lengthy rants. No, I think enough of us have a vested interest in seeing this kind of thing done properly. > (I see that Boulevard isn't available in Chicago, so I reckon I'll be > bringing another trunkful to YAPC2008. Not sure about Flying Monkey, but > probably that too.) Seriously, I still think about that Boulevard beer from time to time. I wonder if my local beverage distributor could acquire a case for me. -- rjbs
the big todo list, posted
http://emailproject.perl.org/wiki/Big_Todo_List I've dumped my previous mail on this topic to the wiki, in response to the occasional, "What can I do?" mails and privmsgs. So far, these haven't led to much, but... well, a fellow can hope. -- rjbs
new Email::Abstract!
At long last (and about 3 months past due, says my OmniFocus) I have released a new stable release of Email::Abstract. This release should be significantly saner, with better facilities for a plugin to declare when it will work and when it should be ignored. Email::Abstract is likely to be the email representation passed around in Email::Sender, which should be the next thing to get some attention. Please let me know if you have any problems or concerns about it. -- rjbs
pep and 5.005
I was mildly surprised when I saw Michael Schwern's recent announcement that he would no longer be supporting 5.005. I was happy to see it, though, because my plan has been to drop 5.005 support, with perl 5.010 coming out. This will mean, at the very least, that 'warnings' and 'our' will no longer be taboo. I'm not going to go out of my way to break things, but as modules get worked on, I will no longer shy away from breaking compatibility with perls more than seven years old. Email::Date::Format, today's new module, requires 5.006. In a few days, MIME::Lite will require it (to drop Email::Date) and I'm sure more will follow. That's all -- I hope everyone's upgraded sometime in the last seven years. -- rjbs
(non-)progress on Email::Base and Email::Sender
I'm tired of trying to fix hard-to-fix issues in Email::Send, and tired of
saying, "Yeah yeah, Email::Sender someday soon blah blah."
So, I'm doing a bit more work on it. Here are my current thoughts:
Email::Base is good enough for now. All it does is provide ->throw, which is
enough. Someday, I imagine there will be more to it.
The big thing it doesn't do that we talked about is a generic constructor and
attributes. This isn't a big thing, but it's starting to mildly irritate me as
I work on Email::Sender. I don't want to just "bless $arg => $class", and I
don't want to say, "all guts are available to all subclasses" or "stick your
subclass's guts in $self->{__PACKAGE__}. These all suck.
Also, I've refactored "args for the Sender" from "envelope data." Given a mail
with a "prec" send-time option, this is still correct:
$sender->send($email, { to => $rcpt, from => $sender, prec => 'bulk' });
...but it is turned into:
$sender->send_email(
$email,
{ to => $rcpt, from => $sender },
{ prec => 'bulk' },
);
Now, you end up writing:
sub send_email {
my ($self, $email, $env, $arg) = @_;
...
}
Blech. What I really want is something we've long talked about internally at
work, and something that DaveO and I talked about once or twice at YAPC:
messages with attached envelopes. Email + Envelope = Delivery. The you say:
$sender->send($email, { to => $rcpt, from => $sender, prec => 'bulk' });
...and it becomes:
$sender->send($delivery, { prec => 'bulk' });
If you already have a Delivery ready to go, you pass it to the $sender->send to
begin with.
The way I see it, a Delivery does the Email and Envelope roles. It has-a Email
and has-a Envelope, and delegates the roles to them. This (plus the pair of
fuzzy antler headgear I received for Christmas) is making me start to decide
that I'm going to say "screw it" and use Moose. If Email::Sender is a Moose
object, it becomes very simple to deal with the attribute issues. I also think
(but cannot yet prove) that it will make Email::Sender::Wrapper trivial to do
away with, letting Email::Sender::Failable operate directly on concrete Sender
classes.
I will start work on this in the next week, and it should be very easy. If you
have objections, voice them now.
My only concern at the moment is that I want the Email role to be easy to grow
in the future. I'll spill my guts on some of that now:
I want method naming to be uniform across all of these classes, and I think it
may have to be set/get. Here is why: headers. It is very convenient to say:
$email->header($header_name);
This is also convenient:
$email->header($header_name => $value);
This starts to get weird:
$email->header($header_name => $value, $value2);
This sucks:
$email->header($header_name => undef);
I'm also not crazy about "well you use set to set it to anything except for
nothing, which requires delete." I could be convinced, though. There's always
"setting multiple values requires an array ref, and [] clears." I don't know,
I welcome thoughts on the matter.
There will be a EmailHeader role, I think. I'm not sure how it will work yet,
either. I will probably steal or at least think about stealing from Mail::Box,
which does some cool stuff with how bodies and headers interact.
Ok, this message is now officially too long. I look forward to hearing your
thoughts.
--
rjbs
Re: Email::Store is dead! Long live Email::Store!
* Justin Simoni <[EMAIL PROTECTED]> [2007-12-19T20:14:58] > You know, I could really use some help with Dada Mail - the last time > this thread was active, my message I sent telling people about it was > never posted. > > Anyways, the next alpha release of Dada Mail is looking pretty killer. > If anyone is interested in helping with development, let me know what > your pet project may be. I had a glance at it, but I need to look more. Maybe I can help make it CPANable. I work at IC Group, which runs (among other services) Listbox.com, which is a really nice MLM. I'm looking to abstract and publish parts of that system as a generic MLM framework, of which Listbox would be one implementation. I'm hoping that I can release a lot of the cool stuff we have, without the impossible (and unappealing) task of trying to make Listbox itself a complete, released free software product. I will post about that work shortly. Maybe it will turn out that Dada and Listbox (and other things) are similar enough that building this framework really will be useful. > One of the projects I'd really love help on is getting a proper, make, > make test, make install distribution and having it hosted on CPAN. > Right now, I've just been working on the test suite, which has about > 5,000 tests already. I think that if you make it a CPAN distribution, you will get more people to help faster. Being able to say "install this dist" and have it Just Go is a huge time-saver for potential hackers, in my experience. I know that *I* certainly give up on a lot of Perl code when I see it isn't packaged standardwise. > but I've managed to do quite a heap of work already. A lot of the code > isn't up to the pep spec, but a lot of the code was written before > pep's recommended CPAN modules existed. That's OK, a lot of PEP's code isn't up to PEP standards. Actually, a lot of it wakes me up at night in a cold sweat. Improving it is half the fun. (The other half is ... well, I'll figure it out sometime.) > Another project may be to get it working with those modules - for example, > replacing the MIME-Tools stuff with other things. One of the main goals of > Dada Mail is to have it installed without needing any XS perl modules, so > people who do not have CPAN (or don't know how to use) or a compiler can > install it. MIME-tools is awful in some ways that I know about and some that I don't, but it's also got a LOT of edge on Email::MIME in many places. I'd give a look at Mail::Message and see if it looks worth upgrading. It very well may not. P.S., Hugo Ball rules. -- rjbs
Email::Corpus
Two emails. First, a sample of what I think it will look like:
use Email::Corpus;
use Test::More;
use Hypthetical::DKIM::Tester qw(check_dkim);
my $collection = Email::Corpus->search({
package => [ -Core, -DKIM ],
name=> $name,
tags=> [ qw(mime tnef) ],
filter => sub { exists $_->meta->{dkim} },
});
plan tests => $collection->count;
while (my $email = $collection->next) {
my ($pkg, $guid) = map { $email->$_ } qw(package guid);
is(
check_dkim($email->abstract),
$email->meta->{dkim},
"expected dkim result on $guid in $pkg",
);
}
--
rjbs
Email::Corpus building
To build a corpus distribution, you'll use Module::Install::EmailCorpus. It will cause 'make' to build an SQLite database out of the contents of ./corpus (see below) and will cause that file ot be installed to the package's sharedir. ./corpus will contain one directory per message, each of which will contain a "message" file and a "meta" file. The "meta" file will contain a single YAML document with a sequence of data about the message. Every message will have a guid. (Module::Install::EmailCorpus will, I hope, provide a 'make guidify' to add guids where needed, and will refuse to 'make dist' if not every message has one.) A sample meta file might be: --- guid: 3F2504E0-4F89-11D3-9A0C-0305E82C3301 tags: - mime - crlf:lf - 8bit Email::Corpus::Magic: magic: 1 moon: full The meaning of package-named mapping entries are left up to the owner of the namespace to define, by convention. -- rjbs
Re: IMAP and message modification
* "Roderick A. Anderson" <[EMAIL PROTECTED]> [2008-01-14T17:09:38] > I've read (most of) the IMAP RFCs and searched and looked at many of the > CPAN modules that deal with IMAP but I'm having no success finding what > I'm looking for. I'm probably using the wrong terminology. Have you tried using Mail::Box::IMAP4 to get the message, alter it, and then call ->update on the box? -- rjbs
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-17T19:50:52] > The MD::BounceParser seemed to bee a bit disorganized and had seems to > have the assumption that you only would want to use it for finding out > email-addresses to remove from send-lists. That is definitely the reason it was made. It was also written ages ago, and it's difficult to extend to add things like, "reason = they think it's spam." > Depending of what you want to do, you will treat these cases differently. > A hard bounce is a response that the mailbox no longer exist. A soft > bounce is a mailbox over quota or some other thing that may not be a > permanent condition, but could be. A transient bounce is just a status > report and not a failure. But I still want to catch those transient > error messages. They should certainly not go into the request queue, but > they should still be parsed and used to update the status of the > original email and receiving user. This can be done then there is a > custom built web interface for the email handling. "Hard" and "soft" are definitely well-established terms for describing bounces. I think that "transient bounce" is a weird choice, though. It's not a bounce, and if it was, a transient bounce would sound, to me, like a soft bounce. It's just a DSN that isn't a bounce reporting DSN. I agree, of course, that understanding these is a good idea. > I have set up Email::Classifier. I just took all the code of > MD::BounceParser and converted it. It uses modules that loads on > demand. Most of the code ended up in the Email::Classifier::Bounce > module, that still needs a lot of cleaning up. I look forward to having a look at it! I will try to do so today or tomorrow, as time permits. Thanks very much! -- rjbs pgp5E1CeCNFZv.pgp Description: PGP signature
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-18T16:45:32]
> http://jonas.liljegren.org/perl/dist/Email-Classifier-0.01.tar.gz
I finally got around to flipping through this, tonight!
I think my main concern relates to the way that one would use Classifier, and
what it would return. I imagine this as a very basic use case:
my $classifier = Email::Classifier->new({
classifiers => [ ... ],
...
};
my $report = $classifier->classify($email);
if ($report->type eq 'bounce') {
my $details = $report->details;
log("got bounce from $details->{remote_mta});
if ($details->{status_code} =~ /^5/) {
...
}
}
A classifier returns a report, which has a globally fixed API. It can provide
details, which are defined by the winning classifier. The report is generated
by the specific classifiers set up in construction. The first one to issue a
report wins. If no classifier matches, we return false.
This will handle most use cases, I think.
It will also be easy to say:
my @reports = $classifier->all_classifications($email);
...to get every report, in the event that we think ones after the first hit
will be useful.
If a $report has one and only one type, we could say:
my $report_set = $classifier->all_classifications($email);
my @bounce_reports = $reports_set->of_type('bounce');
...but now I'm getting into exotica. I think the really nice simple bit of
design is that Email::Classifier runs an email through a set of specified
classifiers, stops when it gets a hit, and returns an object with a globally
constant API.
It's easy, then, to say that the classifiers for Email::Classifier are, in
turn, Email::Classifiers. So, Email::Classifier::Bounce is just an
Email::Classifier built up of all its Email::Classifier-compliant plugins for
specific bounce formats.
This also avoids all the is_* methods, conflict detection, AUTOLOAD, and
typeglob wrangling in the code you posted.
I also think that totally ditching Mail::DeliveryStatus::BounceParser is a good
start: it will be a nightmare to fuss with or extend, even in the form you've
pruned it down to.
So, if I am not full of crap, the two big tasks are:
1. implement Email::Classifier, which should be really easy
2. implement various useful classifiers, starting with some bounce processors
A good precursor to (2) is to build up a good corpus of messages.
Thoughts?
--
rjbs
Re: Email::Classifier
* Ricardo SIGNES <[EMAIL PROTECTED]> [2008-02-21T23:38:18]
> I think my main concern relates to the way that one would use Classifier, and
> what it would return. I imagine this as a very basic use case:
>
> my $classifier = Email::Classifier->new({
> classifiers => [ ... ],
> ...
> };
Rather than just wave my hands and say, "Uh, something like this," I have put
together a quick implementation of Email::Classifier as I described it.
Something not implemented, from a post in July by Dave O:
> my $r = $parser->parse( $email_abstract );
> if( $r->is_gpg_signed() ) { # let plugins provide new methods to
> # result object (inheritance? Sub::Exporter?)
> if( $r->valid_signature_by('0x12345678') ) {
> # Yes, we fully trust the message now
> eval $email_abstract->get_body();
...the ability to add report methods. I think that needs more thought.
Tarball attached.
--
rjbs
Email-Classifier-0.001.tar.gz
Description: application/tar-gz
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-25T05:04:56]
> I still want to have methods dependent on the type of classifier. The
> orig_message_id may have to scan the message, searching for the id. But
> it should not have to do so if the message_id isn't going to be used.
>
> Other examples are orig_message_head that should try to extract the
> message_head from what ever format used. Maby giving access to classifier?
>
> my $report = $classifier->classify($email);
> if( $report->type eq 'dsn' )
> {
> print "DSN for ".$report->obj->orig_message_id;
> }
Under the code I sent, this could be implemented as:
my $report = $classifier->classify($email);
if( $report->type eq 'dsn' )
{
print "DSN for " . $report->details->{orig_message_id};
}
There's no need to keep the classifier around or have a reference to it. The
classifier would just end up doing:
if ($is_bounce) {
return $self->match({
orig_message_id => $msg_id,
type=> $computed_type,
...
});
}
--
rjbs
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-25T08:25:54]
> I want to get it on demand. Not extracting all possible information just in
> case it may bee needed. It could be hundreds of different things.
>
> Ok. So I could let $msg_id be a object that computes itself on
> stringification.
My first thought is YAGNI: You Aren't Gonna Need It. That is, I predict that
there are not hundreds of things that any given classifier is likely to report
on, only a few, which would not be too expensive to compute up front.
Let's say I'm wrong: I could see this being nice and simple:
$report->details->{helper}->orig_msg_id;
Where the "helper" is an object that can compute a bunch of stuff as needed,
rather than using a lot of thunks.
I don't even mind if "details" is sometimes set to an object that can be
accessed as a hash, so that you could say $report->details->orig_msg_id; In
other words, the semantics of "details" can change a bit, but the semantics of
"report" should not.
Here is my concern in more detail:
my $classifier = E::C->new({
...
classifiers => [ qw(-Bounce -SpamAssassin -Autoreply -ClamAV -Cloudmark) ],
...
});
my $report = $classifier->classify($email);
if ($report->spam_score > 5) {
...
}
Well, first, where does "spam_score" come from? I see a few options:
1. Each classifier can declare that its reports are of a specific class.
So, E::C::SpamAssassin reports with Email::Classifier::Report::SpamAssassin.
That has a "spam_score" method.
The problem here is that if the result was a Bounce report, there will be no
spam_score method and now you've thrown a method missing exception. Having
to eval every method call on $report seems awful.
2. Each classifier can import more methods into the report class.
So, when loaded, E::C::SA adds a spam_score method to
Email::Classifier::Report. Now every report has a spam_score method.
Unfortunately, -Cloudmark wants to also. There is a conflict, so we can't
use the same name.
Or maybe we can: the methods are created and they do nothing unless the
report is of a class known to handle the method. This means that every
added method needs to be able to return undef (or something else) if the
report doesn't handle it. That's weird to begin with. Another problem is
that SpamAssassin and Cloudmark may both return spam_score, but with very
different meanings. A "10" on SpamAssassin is quite high, but quite low for
Cloudmark.
So, maybe we force all plugins to use unique names. spamassassin_spam_score
and cloudmark_spam_score. We still might see conflicts, especially if we
have *two* SpamAssassin classifiers loaded with different config:
# This uses the second term as a classifier name, as dave0 originally
# suggested.
classifiers => [
[ -SpamAssassin => lax=> { hits_required => 10 } ],
[ -SpamAssassin => strict => { hits_required => 5 } ],
],
Well, now we could prefix all added methods with the moniker:
$report->lax_spamassassin_spam_score
3. Like #2, but optional and isolated:
In other words:
my $classifier = Email::Classifier->new({
...
classifiers => [ ... ]
extended_reports => 1,
});
If extended_reports is set, a new class is created, subclassing
Email::Classifier::Report, with methods mixed in. Other Email::Classifiers
in the same process, then, are not affected. I think this is a necessary
enhancement to option #2.
None of these options, though address the problem of aggregation: the analyze
method, which does not skip rejects or short-circuit at the first match,
returns a ResultSet object which nicely aggregates the result data. It doesn't
have a type or details method, so I guess it may not be a big deal if it
doesn't get any of the other extra methods.
Option #3 strikes me as sort of overly magical, but I can see its appeal.
Anyone else have thoughts?
--
rjbs
Re: validating email addresses
* Dave Howorth <[EMAIL PROTECTED]> [2008-04-17T05:42:59] > I use Email::Valid to check the syntax and MX record. I'm wondering if > there is any best practice for the next steps: > > - verify the existence of the mailbox Don't. Email verification probing is basically abuse, in my opinion. The correct thing to verify is that a person can read the mail sent to the address. In other words: > - send an email requiring a reply The common practice these days is to send an email that has a link in it. Sending one that also accepts a reply is just bonus. I have some code that helps a little with this, but there's not much to it. $user->set_verification_token($random_value); $link = generate_link_with_token($random_value); $email = generate_email_with_link($link); send($email, to => $user); Then when you get a hit at the link with that random value in it, the user is verified. I'm sure this has been implemented a million times on CPAN. My implementation is in Rubric. -- rjbs
Re: Announcement Test::SMTP module released
* Jose Luis Martinez <[EMAIL PROTECTED]> [2008-04-28T14:24:40] > I released a week ago Test::SMTP. This module is a helper for writing > tests for SMTP servers. I basically announced it on the qpsmtpd list, but > I've just found this list, so I'm reannouncing: Thanks for the note. I saw it in the CPAN uploads, and have wanted to check it out, but so far I haven't had the time. -- rjbs
Re: Problem with attachment, 1 becomes 10
* Mikael Bonnier <[EMAIL PROTECTED]> [2008-05-04T20:49:51] > I'm trying to write a program that can send out mails to companies where > I'm interested in working. These mails are the same except that the company > name is switched. My CV should be attached as a pdf-file. Let me rephrase your question: I am trying to spam recuiters. Can you help? No. -- rjbs
new server: some organizational questions:
For boring reasons, I'm moving emailproject.perl.org to a new box. My plan is to just port over the existing Subversion repository and MediaWiki. If you want to demand major changes, this is the time to do it. -- rjbs
Re: parse out the email body
How many times did you send this message to the list?
* "Basil A. Daoust" <[EMAIL PROTECTED]> [2008-06-24T14:59:34]
> I want to be able to read a MIME encoded email file (Thunderbird) and have
> a way of returning just the VIEWABLE body part.
"Viewable" isn't defined by any RFC. What do you mean?
> I tried Email::Simple but I get all the MIME encoding and both the TEXT and
> the HTML parts.
That won't work, because Email::Simple doesn't understand MIME.
> I tried Mail::Internet but then besides everything that Email::Simple gives
> I also get all the header data as part of the body.
That won't work, because Mail::Internet doesn't understand MIME.
> my $obj = Mail::Internet->new([ /\n/, $message ] );my $body =
> $obj->body;
> print @$body,"\n";
> That returned the entire email, headers and everything, so much for
> getting the body.
I think you meant to put a "split" after the [. That's why it put everything
in the body.
You want something like:
my $root = Email::MIME->new($message_text);
my ($html_part) = grep { $_->content_type =~ m{^text/html} } $root->subparts;
my $encoded_html = $html_part->body;
my $html = Encode::decode($charset_from_content_type, $encoded_html);
--
rjbs
Re: parse out the email body
* Basil Daoust <[EMAIL PROTECTED]> [2008-06-25T22:23:07]
> > You want something like:
> >
> > my $root = Email::MIME->new($message_text);
> > my ($html_part) = grep { $_->content_type =~ m{^text/html} }
> > $root->subparts;
> >
> > my $encoded_html = $html_part->body;
> > my $html = Encode::decode($charset_from_content_type, $encoded_html);
> >
> > Thanks, that works though the way you wrote that I thought
> "$charset_from_content_type" was a MIME set value when new was run. Well it
> doesn't appear to be, but no problem.
No, you have to do something like:
use Email::MIME::ContentType;
my $ct = parse_content_type($part->content_type);
my $charset = $ct->{attributes}{charset};
> Now this works and so does pulling "Text/plain". Text plain is what I want,
> so when its available great. I'm guessing that to cleanup text/html, I
> should use something like HTML::Parser?
I don't know what you mean by "cleanup."
--
rjbs
