PEP IRC BBQ

[Ricardo SIGNES]

From now on, I will be findable in #email on irc.perl.org.  I might not be at
my console, but I'll be in the channel.  Show up if you're afraid of sending
email.  (If you're afraid of sending email... maybe PEP can help!)

-- 
rjbs


signature.asc
Description: Digital signature

Re: PEP IRC BBQ

[Ricardo SIGNES]

* Karen J. Cravens [EMAIL PROTECTED] [2006-07-06T14:52:23]
 I deleted the message where you mentioned it, but... is 
 Email::LocalDelivery really still a going concern?  Shouldn't it be (or 
 isn't it already) rolled into Email::Send?  Or am I completely 
 misremembering (I stopped using Email::LD when I switched to SQL-based 
 storage)?

They're different.  Email::Send takes a message and fires it out into the
internet like a circus performer from a cannon.  Email::LocalDelivery drops a
message into a file on your local disk like ice cream from a sad child's cone.

In other words:  Email::Send is like sendmail(1) and Email::LocalDelivery is
like deliver(8).  Any program that wants to write a message to an mbox or
Maildir wants E::LD.  Any program that wants to send mail to another host wants
Email::Send.

I am interested in writing a LocalDelivery mailer for Email::Send, however,
that will send mail to local storage.  Useful!

-- 
rjbs


signature.asc
Description: Digital signature

a barrage of releases!

[Ricardo SIGNES]

Today, I released new versions of:

  Email-Address
  Email-Folder-IMAP
  Email-Folder-IMAPS
  Email-LocalDelivery
  Email-MessageID
  Email-Simple
  Email-Simple-Creator
  Email-Simple-Headers

Over the rest of the week, I'll probably release more Email:: bugfixes.  Once
RT decides that I'm cool enough to close bugs, the total count of Email:: bugs
should start dropping.  Killing bugs and making it clear, through action, that
the Email:: modules are still alive has been my main goal for the last week.
By the end of this week, I think we'll have bugs in Email:: down to a
manageable level, and I'll be really excited about the prospect of making some
progress on new development.

I'll probably dump my thoughts here or on the wiki, or both.  Expect me to
ramble about:

  * fixing Email::Send
  * a simpler Email::Send wrapper
  * a reusable corpus of test messages and an API to it
  * simplifying Email::Abstract use
  * standards for mixins
  * improved usability and ubiquity for Email::Envelope

Dump your ideas, too, and we can hash out what we want to do next!

-- 
rjbs


signature.asc
Description: Digital signature

Re: wiki pagecount eexplosion

[Ricardo SIGNES]

* Dave O'Neill [EMAIL PROTECTED] [2006-07-12T22:02:31]
 Sheer volume aside, it's probably still a good idea.  I don't think
 there's a single-page comprehensive listing of email-related modules
 anywhere else.

I know there are other modules that fit in.  There's the SMS and MMS series...
suggestions welcome.

 Maybe it doesn't make sense to keep that category. Given that there
 would likely be about a hundred or so modules in it, it's probably
 easier to mark the PEP modules as such (whatever we determine that to
 mean), rather than marking non-PEP modules.

I think that's a good idea.

 Earlier today we were talking on IRC about some possible categories.  I
 don't recall quite what was said, but I think the potential starting
 categories were something like this:
   - all CPAN modules that send, receive, parse, or manipulate email
 (basically, Category:Modules as it exists now)
   - modules owned/maintained by PEP
   - modules that PEP recommends (both PEP-maintained and
 not, will change over time)
   - modules that PEP recommends you not use (unmaintained,
 deprecated, or just plain bad)

I think this is an accurate portrait of what we talked about.  I think the
relevant categories will be:

  Modules   - it's a Perl module!
  PEPMaintained - it's in the PEP repository and will follow PEP guidelines
  (yet to be determined; things like, n people review API
  changes)
  PEPEndorsed   - x out of y active PEP contributors think this is a good 
  module for what it does
  SeemsAbandoned - no updates in a long time, despite open bugs
  Superseded - another module provides all of the functionality, better

I've also created these categories:

  HasProblems - the article includes a description of problems that go beyond
simple-to-fix bugs; either they're hard or they're design
issues
  HasIdeas- the article includes a section about ideas for the future

The latter two should be created with templates:  {{Problems}} and {{Ideas}}
will create a section header and category marker.

 At this point, it probably makes sense to define a PEP module as the
 email-handling modules that are a) authored or currently maintained by
 someone participating in PEP and b) don't make other PEP authors wince.
 If that turns out to be unworkable in the future, we can always
 change our minds, but we have to start somewhere.

Works for me!

I'm going to JFDI and stick these categories where I think they belong.  Argue
or extend my ideas as y'all see fit!

-- 
rjbs


pgpJwHSX7zQeb.pgp
Description: PGP signature

Re: possible changes for Mail::DeliveryStatus::BounceParser

[Ricardo SIGNES]

* William Yardley [EMAIL PROTECTED] [2006-08-03T18:09:01]
 I've also been working at building some tests for some of the problems
 we've seen, and building a bigger corpus of emails to use for testing.

I am really looking forward to having a nice body of messages selected for use
as proof that feature X is any good.

 I have most recently been working on messing around with the regexes for
 user_unknown (see changes around line 796), and using 5.1.0, 5.1.1,
 5.1.2 and 5.2.2 errors[1] in the status report as a preferred method of
 determining $report-std_reason over text regexes (see changes around
 line 374).

Excellent; I would much rather trust the data that's supposed to tell us
something than the data whose entrails we have ripped out read.

 I also ripped out some AOL / Hotmail specific hacks which I'm pretty
 sure are way out of date now.

I bet there are more yet to go.

 http://veggiechinese.net/bounce_parser_diff_3.txt
 
 None of these changes are checked in; at this point, I'm just soliciting
 opinions, and hoping some other folks might be willing to test these
 changes.

I'd suggest you check them in; it'll be easier to test them.  Consider making a
branch.  Looking at the code changes, I think checking them in would be good.
They're definitely an improvement, unless I'm missing some obnoxious bug.

 I made a few suggestions (the last 2) at:
 http://emailproject.perl.org/wiki/Mail::DeliveryStatus::BounceParser

bounce unless otherwise is, yes, dumb.  I think that leads to a lot of grief.
It makes sense in the project's original context, when it was known that it
should only be seeing bounces, and the author wanted to be able to exclude some
things.  In production, it's an obnoxious assumption.

It should be easy to make that an option.

I think I'll be deploying these changes, after a bit more testing; I hope to
see some improvements in junk avoidance.

-- 
rjbs


signature.asc
Description: Digital signature

Re: possible changes for Mail::DeliveryStatus::BounceParser

[Ricardo SIGNES]

* William Yardley [EMAIL PROTECTED] [2006-08-03T23:25:31]
  Excellent; I would much rather trust the data that's supposed to tell
  us something than the data whose entrails we have ripped out read.
 
 Yeah. The only problem is what to do when there is some sort of conflict
 between the two bits of data. In theory, you'd hope that people

If we find examples of this, the behavior can be tailored to those specific
examples, rather than remain so general.  I think this is still a win.

 I've been thinking about writing a blocked std_reason for bounces that
 appear to come from spam or virus filters, or other site policy type
 blocks - this might make dealing with those blocks a bit easier for
 organizations... even the most legitimate of lists end up getting
 blocked by someone, somehow.

Sounds good.

 You mentioned something about no_problemo - maybe we should get rid of
 it entirely?

Well, I wish it was something like not_a_bounce, but we can't just change it,
as things probably rely on it.  Adding code to make MBP return something
else based on an option seems like it could introduce bugs with no clear
benefit (beyond feeling less silly when checking std_reason).  What we *can* do
is only return it when std_reason is called on a parsed non-bounce, rather than
actually setting it on the object.  So:

  sub std_reason { return $_[0]-{std_reason} || 'no_problemo' };

(We can make 'get' use methods to get internal values, to make this work.  This
lets us provide more dynamic behavior, which is good.  'get' is a dumb method.)

That would also let us drop -{is_bounce} and have the is_bounce method return
the std_reason, leaving it undef for non-bounces  Same behavior, more
information.

-- 
rjbs
 
 w
 


signature.asc
Description: Digital signature

Re: Email::Simple changes?

[Ricardo SIGNES]

* Ricardo SIGNES [EMAIL PROTECTED] [2007-02-14T16:57:49]
 Oh well!  Please update to 0.211.

Update to 0.213.  0.211 and 0.212 could have problems when delivering the same
message to multiple destinations, which was not properly covered by the test
suite.

-- 
rjbs

Re: Email::Send Redux

[Ricardo SIGNES]

* Dave O'Neill [EMAIL PROTECTED] [2007-02-21T00:07:06]
 On Tue, Feb 20, 2007 at 07:07:43PM -0500, Ricardo SIGNES wrote:
 
  Email::Sender has one main class, the sender.  Email::Sender's send method
  accepts a message and arguments, canonicalizes the arguments, and tries to
  deliver the message.  It returns either a successful delivery report or
  raises an exception.
 
 There's already a Mail::Sender on CPAN... is Email::Sender enough of a
 name collision to cause confusion?

I'm not sure.  I also don't like Email::Mailer, for the same reason.
Email::Delivery::Agent would be okay, but is pretty long.  Then again, if most
people use the simplified interface (like I do, in effect, at work) it may not
be so bad.  Also, it leaves Email::Delivery::Success and so on free.

 Are you planning to define exception classes, or just leave it as a
 free-form die q{Failed for some reason};?

Probably classes.  That's actually what our version at work does.

 might be nice for error handling and recovery.  But, it might also be well
 outside the scope of what you want to do with Email::Simple / Email::Sender.  

Right.  I'm not sure how inclusive I want to be yet.

 Providing an envelope sender and recipient(s) separate from the From/To
 headers of Email::Simple is definitely a good idea.  Maybe we need an
 Email::NotQuiteSoSimple (bad name, but just for discussion's sake...)
 that's basically an Email::Simple with envelope_sender() and
 envelope_recipients() methods added.  They could either be set directly,
 or if left unset, would pull their values from the appropriate
 $self-header('...') values.  

We're working on something like this at work.  The idea is something like:

There is a class for emails, a class for envelopes, and a composite class.  I
don't think Email::Envelope is what we want for the envelope, but I'm not sure.
The current behavior in Sender is to fall back to the to/from headers if not
given explicitly.  With an Email::EmailWithEnvelope, you'd fall back to the
envelope sender/rcpt if no explicit one was given, presumably.

You'd never let your envelope fall back to the email's values, because
they're not related.

  Because the return value of a successful delivery always has a known
  interface (currently I'm calling it Email::Sender::Success, but something
  more like Email::Delivery::Successful seems better, but also silly), and
  because there is at least a small core of guaranteed common arguments,
  senders can be swapped out with a known worst-case level of degredation.
 
 How about Email::Status::Success?  Email::Status could be an abstract
 base class for the return codes and exceptions as well, specifying a
 common API for getting/setting error text and SMTP error codes.

Hm.  Sounds extremely top-level and vague.  That could be good or bad. :)

-- 
rjbs

Email::Simple::Creator

[Ricardo SIGNES]

I thought I'd have a quick run through Email::Simple::Creator to clean up some
of its foibles.  Here are the ones that make me wonder...

The changelog says: 

  1.3 2004-07-05
  Create a message using its local crlf (Casey West).
  Include timezone info in the Date header (Steffen Goeldner).

So, create a message using its local CRLF, but:

  $CRLF= \x0a\x0d;
  sub _add_to_header {
  my ($class, $header, $key, $value) = @_;
  return unless $value;
  ${$header} .= join(: , $key, $value) . $CRLF;
  }

A string is built up with _add_to_header, then passed to Email::Simple-new.
Since it will end with a single $CRLF, the doubled-crlf detector will not
determine that \x0a\x0d is the crlf in question, and will, instead, use \n

The header, though, will actually be line-ended with CRLF.  Email::Simple can
parse this properly, but it seems Wrong.  It also has nothing to do with what
ends up in the body, which is generally passed in as a string with its own,
uninspected linefeeds.

I'm going to do some testing, but I think the right thing to do is to try to
generate a header with CRLF dividing the headers and dividing the head from the
body.  I am not excited at the prospect of re-line-breaking the body, though.

Beyond vague feelings of The Right Thing, is there a strong practical reason
for any particular behavior?

-- 
rjbs

Email::Abstract: big updates; test please!

[Ricardo SIGNES]

So, I spent a few hours today writing more comprehensive tests for
Email::Abstract.  It started when Mark Overmeer noted that he thought there was
a bug in the way Mail::Message objects' newlines were handled.  I had a look at
the tests for Email::Abstract and basically found that there were a huge number
of bugs not exposed by the lousy testing.

I've tried to fix them all, and I've gotten 100% test coverage, with much more
assertive tests.  Here are some of the most important bugs I've addressed:

  * newlines were lost in the bodies of Mail::Message and Mail::Internet objs
  * headers were not unfolded from Mail::Internet and MIME::Entity objects
  * the means of determining the adapter classes were not consistent

I'm a little worried about some of the changes, especially my forcible
unfolding of headers for Mail::Header objects.  Then again, I believe things
are correct NOW and it's hard to imagine anyone relied on this BEFORE.

I want to use Email::Abstract in Email::Sender, so getting this right is
important.  I want to make sure the existing behavior is good before I add
anything else -- and I will definitely add a stream_to (print_to?) method in a
release soon.

So, testing, patches, and / or review appreciated.  It's in svn and on the
CPAN.

-- 
rjbs

Re: Email::Base 0.000

[Ricardo SIGNES]

* Hans Dieter Pearcey [EMAIL PROTECTED] [2007-07-24T22:00:15]
 On Tue, Jul 24, 2007 at 07:46:36AM -0400, Dave O'Neill wrote:
  Using '::Bar::Baz' is probably a better choice.  Most Perl people know
  what a :: prefix means for namespacing, but only Catalyst people will
  find '+Bar::Baz' intuitive.
 
 I think this is worse, honestly; I wrote a whole reply to you assuming that
 you meant ::Bar::Baz would have Email::Exception prepended, because that is
 how I instinctively parsed it.

  -throw('Fully::Qualified::Name');
  -throw('-Under::EE');

(a) hypen means tacked on to previous thing like hyper -active and -trophic
(b) it lets you skip quoting
(c) which means less typing for the common case

  -throw(-Under::EE = { });

I dunno, it's late and I'm about to sleep, but it's a thought.

-- 
rjbs

MIME-Lite in pep svn

[Ricardo SIGNES]

As those of you on pep-checkins saw (sorry for the huge noise), MIME-Lite is
now in the PEP subversion repository.  I've made a new dev release with zero
code changes.

I'm not a fan of MIME-Lite, as it's fairly buggy and not all that lite.  I'm
not really interested in making it the best MIME generator around (though feel
free to do that), but I would like to reduce its bugginess, since a lot of
things use it.

As those of you on pep-bugs know (sorry for the brief outage; my bug aggregator
script is lame), there are about 120 bugs against email-related modules.  31 of
those (that is: 25%) are filed against MIME::Lite.

Testing so that 3.020 can be released would be great.  Commits to fix bugs
would be great.  Be great!

-- 
rjbs

(non-)progress on Email::Base and Email::Sender

[Ricardo SIGNES]

I'm tired of trying to fix hard-to-fix issues in Email::Send, and tired of
saying, Yeah yeah, Email::Sender someday soon blah blah.

So, I'm doing a bit more work on it.  Here are my current thoughts:

Email::Base is good enough for now.  All it does is provide -throw, which is
enough.  Someday, I imagine there will be more to it.

The big thing it doesn't do that we talked about is a generic constructor and
attributes.  This isn't a big thing, but it's starting to mildly irritate me as
I work on Email::Sender.  I don't want to just bless $arg = $class, and I
don't want to say, all guts are available to all subclasses or stick your
subclass's guts in $self-{__PACKAGE__}.  These all suck.

Also, I've refactored args for the Sender from envelope data.  Given a mail
with a prec send-time option, this is still correct:

  $sender-send($email, { to = $rcpt, from = $sender, prec = 'bulk' });

...but it is turned into:

  $sender-send_email(
$email,
{ to = $rcpt, from = $sender },
{ prec = 'bulk' },
  );

Now, you end up writing:

  sub send_email {
my ($self, $email, $env, $arg) = @_;
...
  }

Blech.  What I really want is something we've long talked about internally at
work, and something that DaveO and I talked about once or twice at YAPC:
messages with attached envelopes.  Email + Envelope = Delivery.  The you say:

  $sender-send($email, { to = $rcpt, from = $sender, prec = 'bulk' });

...and it becomes:

  $sender-send($delivery, { prec = 'bulk' });

If you already have a Delivery ready to go, you pass it to the $sender-send to
begin with.

The way I see it, a Delivery does the Email and Envelope roles.  It has-a Email
and has-a Envelope, and delegates the roles to them.  This (plus the pair of
fuzzy antler headgear I received for Christmas) is making me start to decide
that I'm going to say screw it and use Moose.  If Email::Sender is a Moose
object, it becomes very simple to deal with the attribute issues.  I also think
(but cannot yet prove) that it will make Email::Sender::Wrapper trivial to do
away with, letting Email::Sender::Failable operate directly on concrete Sender
classes.

I will start work on this in the next week, and it should be very easy.  If you
have objections, voice them now.

My only concern at the moment is that I want the Email role to be easy to grow
in the future.  I'll spill my guts on some of that now:

I want method naming to be uniform across all of these classes, and I think it
may have to be set/get.  Here is why: headers.  It is very convenient to say:

  $email-header($header_name);

This is also convenient:

  $email-header($header_name = $value);

This starts to get weird:

  $email-header($header_name = $value, $value2);

This sucks:

  $email-header($header_name = undef);

I'm also not crazy about well you use set to set it to anything except for
nothing, which requires delete.  I could be convinced, though.  There's always
setting multiple values requires an array ref, and [] clears.  I don't know,
I welcome thoughts on the matter.

There will be a EmailHeader role, I think.  I'm not sure how it will work yet,
either.  I will probably steal or at least think about stealing from Mail::Box,
which does some cool stuff with how bodies and headers interact.

Ok, this message is now officially too long.  I look forward to hearing your
thoughts.

-- 
rjbs

Re: Email::Classifier

[Ricardo SIGNES]

* Jonas Liljegren [EMAIL PROTECTED] [2008-02-17T19:50:52]
 The MD::BounceParser seemed to bee a bit disorganized and had seems to
 have the assumption that you only would want to use it for finding out
 email-addresses to remove from send-lists.

That is definitely the reason it was made.  It was also written ages ago, and
it's difficult to extend to add things like, reason = they think it's spam.

 Depending of what you want to do, you will treat these cases differently.
 A hard bounce is a response that the mailbox no longer exist. A soft
 bounce is a mailbox over quota or some other thing that may not be a
 permanent condition, but could be. A transient bounce is just a status
 report and not a failure. But I still want to catch those transient
 error messages. They should certainly not go into the request queue, but
 they should still be parsed and used to update the status of the
 original email and receiving user. This can be done then there is a
 custom built web interface for the email handling.

Hard and soft are definitely well-established terms for describing bounces.
I think that transient bounce is a weird choice, though.  It's not a bounce,
and if it was, a transient bounce would sound, to me, like a soft bounce.  It's
just a DSN that isn't a bounce reporting DSN.

I agree, of course, that understanding these is a good idea.

 I have set up Email::Classifier. I just took all the code of
 MD::BounceParser and converted it.  It uses modules that loads on
 demand. Most of the code ended up in the Email::Classifier::Bounce
 module, that still needs a lot of cleaning up.

I look forward to having a look at it!  I will try to do so today or tomorrow,
as time permits.  Thanks very much!

-- 
rjbs


pgp5E1CeCNFZv.pgp
Description: PGP signature

Re: Email::Classifier

[Ricardo SIGNES]

* Ricardo SIGNES [EMAIL PROTECTED] [2008-02-21T23:38:18]
 I think my main concern relates to the way that one would use Classifier, and
 what it would return.  I imagine this as a very basic use case:
 
   my $classifier = Email::Classifier-new({
 classifiers = [ ... ],
 ...
   };

Rather than just wave my hands and say, Uh, something like this, I have put
together a quick implementation of Email::Classifier as I described it.

Something not implemented, from a post in July by Dave O:

   my $r = $parser-parse( $email_abstract );
   if( $r-is_gpg_signed() ) {   # let plugins provide new methods to
 # result object (inheritance? Sub::Exporter?)
   if( $r-valid_signature_by('0x12345678') ) {
   # Yes, we fully trust the message now
   eval $email_abstract-get_body();

...the ability to add report methods.  I think that needs more thought.

Tarball attached.

-- 
rjbs


Email-Classifier-0.001.tar.gz
Description: application/tar-gz

Re: Email::Classifier

[Ricardo SIGNES]

* Jonas Liljegren [EMAIL PROTECTED] [2008-02-25T05:04:56]
 I still want to have methods dependent on the type of classifier. The
 orig_message_id may have to scan the message, searching for the id. But
 it should not have to do so if the message_id isn't going to be used.

 Other examples are orig_message_head that should try to extract the
 message_head from what ever format used. Maby giving access to classifier?

   my $report = $classifier-classify($email);
   if( $report-type eq 'dsn' )
   {
 print DSN for .$report-obj-orig_message_id;
   }

Under the code I sent, this could be implemented as:

   my $report = $classifier-classify($email);
   if( $report-type eq 'dsn' )
   {
 print DSN for  . $report-details-{orig_message_id};
   }

There's no need to keep the classifier around or have a reference to it.  The
classifier would just end up doing:

  if ($is_bounce) {
return $self-match({
  orig_message_id = $msg_id,
  type= $computed_type,
  ...
});
  }

-- 
rjbs

Re: Email::Classifier

[Ricardo SIGNES]

* Jonas Liljegren [EMAIL PROTECTED] [2008-02-25T08:25:54]
 I want to get it on demand. Not extracting all possible information just in 
 case it may bee needed. It could be hundreds of different things.

 Ok. So I could let $msg_id be a object that computes itself on 
 stringification.

My first thought is YAGNI: You Aren't Gonna Need It.  That is, I predict that
there are not hundreds of things that any given classifier is likely to report
on, only a few, which would not be too expensive to compute up front.

Let's say I'm wrong:  I could see this being nice and simple:

  $report-details-{helper}-orig_msg_id;

Where the helper is an object that can compute a bunch of stuff as needed,
rather than using a lot of thunks.

I don't even mind if details is sometimes set to an object that can be
accessed as a hash, so that you could say $report-details-orig_msg_id;  In
other words, the semantics of details can change a bit, but the semantics of
report should not.

Here is my concern in more detail:

  my $classifier = E::C-new({
...
classifiers = [ qw(-Bounce -SpamAssassin -Autoreply -ClamAV -Cloudmark) ],
...
  });

  my $report = $classifier-classify($email);

  if ($report-spam_score  5) {
...
  }

Well, first, where does spam_score come from?  I see a few options:

1. Each classifier can declare that its reports are of a specific class.

  So, E::C::SpamAssassin reports with Email::Classifier::Report::SpamAssassin.
  That has a spam_score method.

  The problem here is that if the result was a Bounce report, there will be no
  spam_score method and now you've thrown a method missing exception.  Having
  to eval every method call on $report seems awful.

2. Each classifier can import more methods into the report class.

  So, when loaded, E::C::SA adds a spam_score method to
  Email::Classifier::Report.  Now every report has a spam_score method.
  Unfortunately, -Cloudmark wants to also.  There is a conflict, so we can't
  use the same name.

  Or maybe we can: the methods are created and they do nothing unless the
  report is of a class known to handle the method.  This means that every
  added method needs to be able to return undef (or something else) if the
  report doesn't handle it.  That's weird to begin with.  Another problem is
  that SpamAssassin and Cloudmark may both return spam_score, but with very
  different meanings.  A 10 on SpamAssassin is quite high, but quite low for
  Cloudmark.

  So, maybe we force all plugins to use unique names.  spamassassin_spam_score
  and cloudmark_spam_score.  We still might see conflicts, especially if we
  have *two* SpamAssassin classifiers loaded with different config:

# This uses the second term as a classifier name, as dave0 originally
# suggested.
classifiers = [
  [ -SpamAssassin = lax= { hits_required = 10 } ],
  [ -SpamAssassin = strict = { hits_required =  5 } ],
],

  Well, now we could prefix all added methods with the moniker:

$report-lax_spamassassin_spam_score

3. Like #2, but optional and isolated:

  In other words:

my $classifier = Email::Classifier-new({
  ...
  classifiers  = [ ... ]
  extended_reports = 1,
});

  If extended_reports is set, a new class is created, subclassing
  Email::Classifier::Report, with methods mixed in.  Other Email::Classifiers
  in the same process, then, are not affected.  I think this is a necessary
  enhancement to option #2.

None of these options, though address the problem of aggregation:  the analyze
method, which does not skip rejects or short-circuit at the first match,
returns a ResultSet object which nicely aggregates the result data.  It doesn't
have a type or details method, so I guess it may not be a big deal if it
doesn't get any of the other extra methods.

Option #3 strikes me as sort of overly magical, but I can see its appeal.

Anyone else have thoughts?

-- 
rjbs

Re: validating email addresses

[Ricardo SIGNES]

* Dave Howorth [EMAIL PROTECTED] [2008-04-17T05:42:59]
 I use Email::Valid to check the syntax and MX record. I'm wondering if
 there is any best practice for the next steps:
 
 - verify the existence of the mailbox

Don't.  Email verification probing is basically abuse, in my opinion.  The
correct thing to verify is that a person can read the mail sent to the
address.  In other words:

 - send an email requiring a reply

The common practice these days is to send an email that has a link in it.
Sending one that also accepts a reply is just bonus.

I have some code that helps a little with this, but there's not much to it.

  $user-set_verification_token($random_value);

  $link = generate_link_with_token($random_value);

  $email = generate_email_with_link($link);

  send($email, to = $user);

Then when you get a hit at the link with that random value in it, the user is
verified.  I'm sure this has been implemented a million times on CPAN.  My
implementation is in Rubric.

-- 
rjbs

bug in MIME::Entity make_singlepart

[Ricardo SIGNES]

I thought I'd cc the list since this is sort of a weird, fun bug.

Sometimes, when collapsing a message into single part, the C-T is horked up.

It starts as:

  Content-Type: multipart/related; boundary=xyzzy; type=foo

...and ends as:

  Content-Type: text/plain; boundary=xyzzy; type=foo

Erk!  Boundary?  Type?  What?

Demonstration attached.

-- 
rjbs

Re: bug in MIME::Entity make_singlepart

[Ricardo SIGNES]

* Dave O'Neill [EMAIL PROTECTED] [2008-06-27T10:30:10]
 On Thu, Jun 26, 2008 at 01:37:09PM -0400, Ricardo SIGNES wrote:
Content-Type: multipart/related; boundary=xyzzy; type=foo
Content-Type: text/plain; boundary=xyzzy; type=foo
 
 The docs for make_singlepart say Also crunches 0-part multiparts into
 singleparts, so the first option is probably the safest for backwards
 compatibility.  However, it does seem nonsensical to convert an empty
 multipart/* part into a text/plain.  Is this a case that's frequently
 encountered?  If not, maybe it's OK to change that behavior.

I think that it's documented, so it should keep working.  If nobody has asked
for it to throw an error, it either isn't coming up or it's coming up and
people accept it.

I'd suggest adding an error-generating branch (with a different method name or
an arg) or just removing the entire Content-* header set (since there is no
content!) but I wouldn't need to use it, and who knows if anyone else would.

I think the only problematic behavior is the bizarre (and presumably illegal)
leftover parameters.

-- 
rjbs

Email::Classifier: not dead yet

[Ricardo SIGNES]

I need to stop sitting on good ideas!

Way back in February, someone posted saying, how about that email classifier
we want to replace the awful awful BounceParser?  I replied with some of my
own ideas, including an implementation.  Then I did nothing.

  http://www.nntp.perl.org/group/perl.pep/2008/02/msg288.html

I finally need Email::Classifier for something else, and I brushed off the
code.  I realized that as written, only the tests were email-specific.  I made
it a generic thing classifier, improved the interface just a tiny bit, and made
it use Moose.

You can find Classifier here:

  http://github.com/rjbs/classifier/tree/master

I may sit on it for another week or so, but unless I get some feedback on the
order of no, stop, this is crazy! I am going to release it and start using it
for things.  Your input is hereby requested!

-- 
rjbs

Re: Email::Classifier: not dead yet

[Ricardo SIGNES]

* Simon Wistow [EMAIL PROTECTED] [2008-12-01T20:45:53]
 On Mon, Dec 01, 2008 at 07:15:20PM -0500, Ricardo SIGNES said:
  I may sit on it for another week or so, but unless I get some feedback on
  the order of no, stop, this is crazy! I am going to release it and start
  using it for things.  Your input is hereby requested!
 
 This is more a point of philosophy and should probably be rephrased as a 
 question but - the original goal of the Email project when I helped 
 start it was to be as fast and lightweight as possible with as few 
 non-core dependencies as could possibly be gotten away with[*]. 

I have long been toying with putting much of this stuff under something other
than Email, but it's sort of a moot point.  There is plenty of garbage under
Email::, of varying sorts.  There's old-school let's be simple Email:: code
that is so badly designed as to be unsalvageable and there's code from people
who know nothing of the origins of Email:: and just thought it was the right
place to put their big, reasonable email-related code.

I think that having a top-level namespace that was supposed to all be for one
related project's set of code was a failure when (a) a common noun was chosen
(b) the actual project code was not built to clearly all go together.

That said, I do keep that original goal in mind, and the email-specific
Classifier code may well be Mail::Classifier or something.  Until today,
Email::Classifier's code was not Moosified.  Making it Moose saved me a lot of
crap code.  Making it generic is also going to be useful, I know already.

 As far as I can tell from a quick glance Classifier could be trivially 
 rewritten to use absolutely no non-core dependencies - I know Moose is 
 the flavour du jour but I'm still sad to the original goal abandoned.

It's true, it could be, but extending it would be more of a pain, I'd need to
test much more of its behavior, etc etc etc.

I resisted Moose at work for quite a long time, maybe a year or more, before
giving in and realizing how incredibly useful it is and how much time it saves
me.  Then there's Mouse, which gets you quite a lot of that, with no prereqs.

Anyway, I don't mean to sound touchy, but I promise you that I resisted using
Moose for anything in E/mail for a -long- time.  I still plan to avoid it for
the Email::Send replacement.

My current guiding principle is If you want to use it in some random code that
deals with email, be as slim as possible.  If you are going to use it for
something that's really just about doing email and doing it well, don't mess
around.

If that means I'm supposed to use Mail::, then... so be it.  It just seems like
at some point Mail will start relying on Email and nobody will get it but the
cabal.

 But hey, this is your baby now (and has been for a long time) so I 
 suppose I have no right to grouse. 

Your grousing is always welcome!

-- 
rjbs

Email::Sender: also not dead yet

[Ricardo SIGNES]

The first thing that got me involved in PEP was Email::Send, which was largely
unusable for serious email-related work becase it offered no mechanism for
specifying an envelope sender or recipient.  It had a bunch of other problems,
some of which, we felt made fixing Email::Send in a backwards-compatible way a
non-starter.

Internally, at Pobox, we produced something called Email::Send::Mailer, which
was a base class for Email::Send mail-sending plugins.  It helped solve the
problems, but required non-backwards-compatible changes to Email::Send and was,
generally, a pretty grody workaround.

Immediatley, we started talking about turning that work into a better
Email::Send replacement, which we called (in the office and on the mailing
list) Email::Sender.

After a lot of delays, I've gotten Email::Sender into a state where it's nearly
ready for release.  I need to do a few significant things, but it's mostly
ready to go.

You can see it here: http://github.com/rjbs/email-sender

Right now, it uses Mouse, unless Moose is already loaded.  Mouse has zero
prerequisites, so this is no more of an imposition than, say, Class::Accessor.
Still, if it's going to ruffle too many feathers, I will release this code as
Mail::Xmitter or something.

Things that remain to do:

  * write up SMTP tests that can be used against different servers
  * prune no-longer-needed prereqs (see Makefile.PL for notes)
  * move SMTP_X, the persistent SMTP mailer, to xlib until it works
  * test mbox delivery
  * improve reporting of individual failures in sending, test it

I welcome thoughts, criticism, suggestions, or bug reports.

-- 
rjbs

Email::Sender API approaches stability

[Ricardo SIGNES]

I don't doubt that I'll hit a few glitches on the way to finality, but
Email::Sender's API is getting pretty close to usable.  Check it out and have a
go:  http://github.com/rjbs/email-sender

Right now, normal usage is something like this:

  my $sender =  Email::Sender::Transport::Sendmail-new;

  my $result = eval {
$sender-send(
  $email,
  {
to   = '[EMAIL PROTECTED]',
from = '[EMAIL PROTECTED]',
  },
);
  };

  my $error = @_;

$result is either false or an Email::Sender::Success.  If it's false, $error
had better be an Email::Sender::Failure.  Every failure has a 'message' and an
array of 'recipients' and may have a 'code'.

Sometimes the failure is a ::Multi, meaning that while sending was aborted,
results were mixed.  It contains other failures.  They might all be of one
subclass of Failure (Permanent or Temporary, for example) or there might only
be one failure.  Multi will have one contained failure when, for example:

  * four out of five recipients were accepted
  * ...but the SMTP transport is set to require 100% acceptance (default)

This means that it is generally safe to say:

  $sender-send($email, $env);

...and let things fail if the email was not sent.  This won't always be true,
though, when there exist transports that are not capable of atomic transaction,
like LMTP.  They will be respond to something useful with -does, probably.

Anyway, if you really know what you're doing, you can also:

  my $sender = E::S::T::SMTP-new({
host = 'mx.your-isp',
allow_partial_success = 1,
  });

  my $succ = $sender-send($email, $env);

In the event that 0x100% of your senders are accepted, it will NOT abort the
SMTP conversation before DATA and will return an
Email::Sender::Success::Partial, which has a 'failure' attribute that contains
a Multi.

  for my $addr ($succ-failure-recipients) {
unsubscribe($addr);
  }

I plan to write Email::Sender::Simple at some point to provide an interface a
bit more like Email::Send's, complete with envelope defaults and true/false
values with dying reserved for void context.  Or something.  I haven't given
that much thought yet.

Please let me know if you have feedback, as I'm definitely making progres
toward calling this ready for upload (once there are docs (and more tests)).

-- 
rjbs

Email::Sender 0.000 released

[Ricardo SIGNES]

I still consider it fair game for big changes.

I will be breaking it into a few dists soon.

http://rjbs.manxome.org/rubric/entry/1706

-- 
rjbs

Net-Server-Mail repo

[Ricardo SIGNES]

Xavier,

I feel like I already talked to you about this, but I can't find any record of
it, so I'm guessing that I'm wrong.

emailproject.perl.org/svn has been moved (mostly) to github.  One of the few
things I have not dealt with is Net-Server-Mail, because I am not one of its
maintainers.

I see that you made a commit to Subversion that never made it onto the cpan
(0.06 of XFORWARD).  Are you still working on this project?  I'm actually
looking at using it for some LMTP test work.  If you have it in a new repo,
great.  If not, please let me know if you want me to export its history for
you.  I would like to turn off the svn server bits soon.

Finally, if you're not using it any more, I would be happy to take over
maintenance, release the changes, and fix any bugs I find while playing with
it.

Thanks!

-- 
rjbs

Re: Net-Server-Mail repo

[Ricardo SIGNES]

* Xavier x.guim...@free.fr [2009-01-15T01:26:31]
 I have no other repository. I made a few changes on Net::Server::Mail
 since it work for me (more than 1 million message a day) and I have no
 demand to modify something.
 
 I've taken Net::Server::Mail in maintenance few years ago because there
 was a lot of bug not corrected. Now, if you want to contribute, no
 problem for me. I would be happy not to stay alone on this project.
 I know that there is a problem with LMTP using Unix sockets (CPAN/RT)
 but I can't reproduce it.

Ok.  I would like to delete the current Subversion installation on the box
running emailproject.perl.org, and it sounds like you are not likely to set up
a repo for this.

I will import the existing history to GitHub.  If you'd rather I don't, just
say so and I will either not, or I will delete what I've already done.

Once I've done that, I may check in some changes.  If nothing else, I found
some documentation problems yesterday.

-- 
rjbs

Re: Net-Server-Mail repo

[Ricardo SIGNES]

* xavier.guim...@laposte.net [2009-01-15T12:31:40]
 no problem for me, you can use GIT and delete svn files.

Excellent.  Earlier today I wrote a backpan-to-git importer.  I imported all
previous releases of Net-Server-Mail to git, then the unreleased changes, then
a few minor fixes:

  http://github.com/rjbs/net-server-mail/tree/master

I will probably be ready for a release in the next few days.  Would you like me
to ask you to cut releases as they're ready, or will you add me as co-maint on
PAUSE?

-- 
rjbs

Email::MIME::Kit released

[Ricardo SIGNES]

After a long time and a bunch of rewriting, Email::MIME::Kit has been released.

Here's what I posted in my journal about it:

Building email messages is a pain.  Even if you use a library to build the
message string for you, you have to know a lot of crap and pay attention to a
lot of details.  If you know what those details are, it's a pain.  If you don't
know what they are, you don't feel any pain until later, when you find out all
the ways things went wrong.

It's more of a pain, too, when trying to set up multipart emails that are sent
all the time, like subscription notifications, welcome messages, reports, and
so on.  I've seen a lot of awful solutions to this problem.  My favorite awful
solution was when I saw some code that took a complete, encoded MIME message
and used it as a template.  You'd have to pass it variables properly encoded
with the right Content-Transfer-Encoding for the part into which the variable
was used, sometimes passing two versions of one variable.  Also, as I recall it
used Python's format operator, so it all looked like a massive printf string.

Then there are all the awful messages I get from vendors who build terrible
MIME messages with broken encodings or text parts poorly converted from HTML
(or empty!).  Also, American Express has ignored my repeated complaints that
while they provide a perfectly legible plaintext part, they do not render it,
so I see all their template variables instead of my information.

We set out to solve this problem internally a while ago, and I think it was a
pretty big success.  It made it very easy to throw together email templates
that were maintainable, comprehensible, and that our web guy could edit easily.

There were some problems, though.  The biggest one, for me, was our code's
reliance on one of YAML's more powerful features: tagging.  We used tags to
describe parts of the message in its YAML-based definition file, but dealing
with YAML tags in Perl is still awful.  Rather than pin our hopes to that
improving, we've replaced that design.

Actually, we've rewritten the entire library.  I think it should be easier to
use, easier to extend, and easier to understand.  I'm very, very excited to
start using it for all our internal messages.  (Once again, this is Pobox
saying, We'd love to release this code, but not until we can rewrite it
entirely based on the lessons we've learned.  Normally I don't like that kind
of thinking, but I thinks it keeps serving us (and the CPAN) well!)

The library is called Email::MIME::Kit.  It can be fairly significantly
customized, but a fairly simple configuration, close to the stock one, works
like this:

You create a directory, which we call a message kit, or mkit, and put a bunch
of files in it.  These files are used in assembling the kit, and the most
important is the manifest.  We decide to write our manifest in YAML:

  ---
  validator: Rx
  renderer:  TT
  header:
  - From:'Customer Support c...@example.com'
  - To:  '[% account.email_address.for_header %]'
  - Subject: 'Your Invoice, Number [% invoice.number %]'
  alternatives:
  - type: text/plain
path: body.txt
  - type: text/html
path: body.html
  attachments:
  - assembler: InvoicePDF
attributes: { filename: invoice.pdf }

When we send a customer a bill, now we can say:

  my $kit = Email::MIME::Kit-new({ source = 'share/msgs/invoice.mkit/' });

  my $email = $kit-assemble({ account = $account, invoice = $invoice });

  $transport-send($email, { ... });

The kit will validate the stash.  Our example uses an
[Rx](http://rjbs.manxome.org/rx/) schema, but writing a validator plugin is
trivial.  With that done, the message is assembled.  The top-level part is
assembled by the standard assembler, but the one attachment consults a custom
assembler that retrieves or generates a PDF from our billing system.

The only thing the designer needs to do is edit the text and HTML files in the
message kit.  These don't need to know anything about the fact that they're
going into email.  They will be properly encoded as needed.  Even the headers
are checked for non-ascii text and encoded as MIME headers (encoded-words) if
needed.

While the templates don't need to know they're going to be part of email, it
can be useful.  This is part of a manifest in Email::MIME::Kit's test suite:

  - container_type: multipart/related
type: text/html
path: better-alternative.html
attributes:
  charset: utf-8
attachments:
  - type: image/jpeg
path: logo.jpg

The HTML template can get the Content-ID of any attachment easily enough,
meaning that you can attach the images you want to reference in your HTML,
rather than rely on web connectivity.  Your template might contain:

  img src=cid:[% cid_for('logo.jpg') %] /

It's worth noting that this is all possible with the standard assembler, but
wordy.  I think we'll probably end up writing an assembler that optimizes for
multipart/related HTML-and-content parts.

Among the 

Re: Sending simple email

[Ricardo SIGNES]

* Bill Moseley mose...@hank.org [2009-06-07T19:07:34]
 On Sat, Jun 06, 2009 at 12:01:40PM -0400, Ricardo SIGNES wrote:
  You have sent this message at an excellent time!  Email::Sender::Simple is
  potentially done, and I've written a quickstart guide, just last night.
  
http://rjbs.manxome.org/tmp/QuickStart.html
 
 Looks pretty good.  Thanks.
 
 Any plans to add simple attachments?  Like I mentioned, one of the
 more common tasks for apps I work on seem to be attaching pdfs and
 spreadsheets.

Well, Email::Sender doesn't deal with attachments at all.  It doesn't know
what's in your message, beyond *maybe* looking at the to/from headers.
Attachment stuff is all handled by Email::MIME.

Can you give me an idea what the Perl you want to write would look like?

  You might like the BBC's Email::MIME::CreateHTML:
  
Email-MIME-CreateHTML
  
  I think it does everything you want.
 
 I looked at that and I don't think it will create the plain text
 version automatically from the html, but I'll take a look again.

You're right, oops.  In these circumstances I do one of three things:

  a) shell out to w3m (ugh!)
  b) only write the plaintext part, in Markdown, and use that to generate both
 (see http://search.cpan.org/dist/Email-MIME-Kit-Assembler-Markdown)
  c) use HTML::FormatText::WithLinks

-- 
rjbs

Re: Sending simple email

[Ricardo SIGNES]

* Roderick A. Anderson raand...@cyber-office.net [2009-06-06T17:50:13]
 Ricardo SIGNES wrote:
 * Bill Moseley mose...@hank.org [2009-06-06T11:34:29]
 I see the recommendation to use Email::Sender, but are there docs
 available?

 You have sent this message at an excellent time!  Email::Sender::Simple is
 potentially done, and I've written a quickstart guide, just last night.

 OK I'm hooked.  How much more potential is there?  :-)

I think I've nearly got all our internal features into Simple.  One that didn't
make it is the archiver, which I will release later as a subclass of Simple.
It basically says after send_email, send AGAIN to ANOTHER transport, but throw
away any failure.  We use this to keep a copy of all the automated messages we
send out without having to write code to do so explicitly.

 I've been trying for a good and simple way to send email from Windows  
 and Linux systems.  So E:S:S looks like it would fill the bill.  In fact  
 I think I've tried and used (with differing success) Email::Sender and  
 friends but never came up with a consistent tool/method that worked for  
 both OS'.

The current dev release of Simple doesn't yet try to use Sendmail transport, it
just starts with SMTP.  The next release will try Sendmail first, and fall back
to SMTP.

Real life experience with using it welcome!

-- 
rjbs

Re: query regarding undocumented feature of Email::Simple

[Ricardo SIGNES]

* Saurabh Hirani saurabh.hir...@gmail.com [2009-06-30T02:07:41]
 So I looked at the source and deduced that if I have
 $email-header_set($headername) i.e I don't give the value to set, as per
 the internal working of the code, it actually deletes the header - which is
 what I want. Is this an intended feature? Because it has not been
 documented. Or is it a side effect of the current code, which might be
 changed later?

That's intentional.  I use it all the time.  I will document it.

 It would really help the users of the Email::* project to know that as many
 wouldn't look at the source. Thanks for developing such a wonderful set of
 modules.

Thanks for the email; it's always nice to know someone's getting some use from
our code!

-- 
rjbs

Re: I do not understand the error CGI that I get CGI using Email:: Simple

[Ricardo Signes]

* fakessh fake...@fakessh.eu [2010-05-03T20:55:20]
 click error
 
 
 when I am not MIME:: Lite 
 I find myself with a CGI error in httpd and browser. 
 I do not understand why

What's the error?

  package PerlWebmail::Message;
  use base qw(MIME::Lite);
  use base qw(Email::Simple);

Extending both those classes at once seems doomed to fail a lot.

  sub mail 
  {
  my ($self, $message) = @_;
  my $obj = Email::Simple-new($message);
  print $obj-as_string;
  $Self-new;
  }

You said $Self instead of $self.

-- 
rjbs

Re: I do not understand the error CGI that I get CGI using Email:: Simple

[Ricardo Signes]

* fakessh fake...@fakessh.eu [2010-05-04T08:20:14]
 [Tue May 04 14:15:11 2010] [error] [client 90.30.250.62] Can't use string
 (fake...@fakessh.eu) as a HASH ref while strict refs in use at
 /usr/lib/perl5/vendor_perl/5.8.8/Email/Simple.pm line 100., referer:

This is a warning, not an error.

 [Tue May 04 14:15:11 2010] [error] [client 90.30.250.62] Premature end of
 script headers: webmail.pl, referer:

It looks like your program just isn't behaving correctly.

   package PerlWebmail::Message;
   use base qw(MIME::Lite);
   use base qw(Email::Simple);
  
  Extending both those classes at once seems doomed to fail a lot.
 
 I can well use Email:: Simple in a CGI

That is not the point.  You are trying to create a class that inherits from two
distinct and contradictory classes.  This will fail.

   sub mail 
   {
   my ($self, $message) = @_;
   my $obj = Email::Simple-new($message);
   print $obj-as_string;
   $Self-new;
   }
  
  You said $Self instead of $self.
 
 typographical error 
 the correcting operation in the class is ok

Don't retype fake code.  Show things that actually fail.

Your program, as shown, makes no sense and will never do anything useful.

-- 
rjbs

Re: Email::Folder stuck at first message

[Ricardo Signes]

* Eddie Rowe webmas...@tarheeloesrescue.org [2010-04-14T11:41:45]
 I have tried to process messages from a Mozilla Thunderbid INBOX file.

Sorry it took me a month to reply.

 [code]
  use Email::Folder;
 
  my $folder = Email::Folder-new(c:/users/eddie/desktop/INBOX.txt);
 
  while( my $message = $folder-next_message ) {
  print join \n, map { $_-header('Date') } $message;
 }
 [/code]

I don't know what format Thunderbird stores stuff in.  Email::Folder guesses
based on the filename, and I don't know what it will guess here.  Probably
stupidly.

Is it mbox format?  Are you *sure*?  If so, use Email::Folder::Mbox directly.

-- 
rjbs

Re: how to add a proper attachment to an email message

[Ricardo Signes]

* fakessh fake...@fakessh.eu [2010-05-13T16:05:31]
 my question is simple
 how to add a proper attachment to an email message
 I use as main modules use Email::Send and use Email::Simple::Creator

You need to use Email::MIME.

-- 
rjbs

Re: do I always need to specify an encoding with Email::MIME?

[Ricardo Signes]

* James Peregrino james_peregr...@harvard.edu [2011-05-02T12:53:46]
 I'm trying to use Email::MIME to send a simple email with a .doc file as an
 attachment.  I receive it fine with Gmail, but my job email chokes on it when
 it tries to scan the attachment for viruses ('UNSCANABLE').
 
 send SMTP = Email::MIME-create(

Please consider using Email::Sender instead of Email::Send.  Seriously, it's
just as easy and so much better in the long run.  Ask me more on IRC if you
like.

 parts = [
 Email::MIME-create(
 body = The exam file is attached to this message.,
 ),
 Email::MIME-create(
 body   = io($EXAM),
 attributes = {
 filename = $EXAM_FILE_NAME,
 content_type = 'application/msword',
 },
 ),
 ],
 

I suggest doing so.  It won't hurt, and it can prevent various forms of
error.  I suggest using quoted-printable for the text and base64 for the
Word document.

-- 
rjbs

Re: Alternative to Mail::IMAPClient that supports proxyauth?

[Ricardo Signes]

* Jesse Thompson jesse.thomp...@doit.wisc.edu [2011-06-23T15:09:24]
 Does anyone know if there is an alternative to Mail::IMAPClient that
 supports proxyauth.

I don't know of any IMAP client worth using much beyond Mail::IMAPClient, which
I use -- and with which proxy auth works.

-- 
rjbs

Re: backwards incompatible asciihood change in Email::Address

[Ricardo Signes]

* Ruslan Zakirov r...@bestpractical.com [2012-01-15T10:24:59]
 On Sun, Jan 15, 2012 at 19:09, Ricardo Signes perl@rjbs.manxome.org 
 wrote:
  I've updated Github's repo with a change to only reject non-ASCII in the
  email address, which really is a problem.  My guess is that you were having
  a problem with the decoded phrase legally containing non-ASCII.
 
 Right guess. Is it legal? I don't think it is legal according to the
 spec to have non-ascii phrase. As far as I recall it should be encoded
 with Q/B.

Sure, but people aren't (I hope) passing the mime-header-encoded content to
-parse.  That will parse, but give you crap.  They should be passing the
decoded character string, at which point the non-ASCII phrase is legal.

i.e., Email::Address parses the header's decoded character data, not its raw
encoded data.

I will make a release in a few hours.

-- 
rjbs

Re: reading webmail inbox from perl

[Ricardo Signes]

* Devireddy, Nagendra ndevire...@ariba.com [2014-01-28T08:40:18]
 Imap

I suggest using Mail::IMAPClient.  It's easy to use and fairly well documented.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Derrivative code

[Ricardo Signes]

* Geoffrey Leach ge...@hughes.net [2014-07-05T18:30:19]
 I'm  at work on MH.pm, which I hope to submit to CPAN as
 Email::LocalDelivery:MH.pm. The code is based on
 Email::LocalDelivery::Maildir.pm
 
 Question, in such a case is there a preference for maintaining the format,
 naming conventions, etc. of the original code? 

I don't think it matters, as long as the interface is compatible.

FWIW, I am soon going to abandon maintenance of Email::LocalDelivery.  I only
have one line of code still using it.  For all other things, I now use
Email::Sender.

-- 
rjbs


signature.asc
Description: Digital signature

Re: things need co-maint

[Ricardo Signes]

* 'lesleyb' lesl...@herlug.org.uk [2014-07-30T06:58:14]
 On Tue, Jul 29, 2014 at 07:03:39AM -0400, Ricardo Signes wrote:
  There are a lot of email modules that I don't use and am not being very
  vigilant about maintaining.  In some cases, I can chown them to HANDOFF on
  PAUSE and in other cases, I'm just comaint and can't do anything but declare
  that future releases from me are unlikely.
  
  These modules include: [...]
 
 So what would you like to see done with these modules? 

I want people to realize that I am not going to keep doing maintenance, and
that if they need ongoing maintenance, they will need to pick up the burden or
go through the other listed maintainers.

-- 
rjbs


signature.asc
Description: Digital signature

Re: things need co-maint

[Ricardo Signes]

* Geoffrey Leach ge...@hughes.net [2014-08-19T19:54:44]
 Does this imply the passing of the Perl Email Project? Perhaps you ncould
 elaborate.

The PEP, as far as I am concerned, is a non-thing.  It is a name describing
nothing that exists.  In about five years, the only thing that has happened
associated with its name have been releases of software once under its
umbrella, done by me, linking to a single static web page.

If someone wants to do something with the name, great.  Otherwise, nothing is
being killed off that was not already moldering.

-- 
rjbs


signature.asc
Description: Digital signature

Re: things need co-maint

[Ricardo Signes]

* Erik Logtenberg e...@logtenberg.eu [2014-08-20T06:03:14]
 Of this list of modules I use, Email-Send and MIME-Lite are on your list
 of modules that need a new (co)maintainer.

I urge you to consider using Email::Sender in place of Email::Send.  It is a
significant upgrade to testability and extensibility.

 Could you incidate if these modules currently have other maintainers,
 and if there are open bugs (i.e. there is already maintenance that needs
 to be done).

MIME-Lite had about 125 tickets when I took up comaint, and is now down to
just 51.  I am only a co-maintainer on it, but have done the only releases in
the last nine years.  It may be that the other maintainers would like to do
more work on it, but I would be somewhat surprised.

Email-Send has no open tickets.

-- 
rjbs


signature.asc
Description: Digital signature

Email::MIME::Kit v3

[Ricardo Signes]

Ever since its early releases, Email::MIME::Kit had a big problem.  It screwed
up encodings.  Specifically, imagine his manifest (I'm kinda skipping some
required junk):

  # manifest.yaml
  renderer: TemplateToolkit
  headers:
- Subject: Message for [% name %]
  alternatives:
- type: text/plain
  path: body.txt
- type: text/html
  path: body.html

The manifest turns into a data structure before it's used, and the subject
header is a text string that, later, will get encoded into MIME encoded-words
on the assumption that it's all Unicode text.

The files on disk are read with :raw, then filled in as-is, and trusted to
already be UTF-8.

If your customer's name is Распутин, strangely enough, you're okay.  The header
handling encodes it properly, and the wide characters (because Cyrillic
codepoints are all above U+00FF) turn into UTF-8 with a warning.  On the other
hand, for some trouble, consider Ævar Arnfjörð Bjarmason.  All those codepoints
are below U+0100, so the non-ASCII ones are encoded directly, and you end up
with =C6 (Æ) in your quoted-printable body instead of =C3=86 (Æ UTF-8 encoded).

Now, you're probably actually okay.  Your email is not correct, but email
clients are good at dealing with your (read: my) stupid mistakes.  If your
email part says it's UTF-8 but it's actually Latin-1, mail clients will usually
do the right thing.

The big problem is when you've got both Ævar Arnfjörð Bjarmason and Распутин
both in your email.  Your body is a mish mash of Latin-1 and UTF-8 data.

In Email::MIME::Kit v3, templates (or non-template bodies) loaded from disk are
— if and only if they're for text/* parts — decoded into text and then, when
the email is assembled, it's encoded by Email::MIME's usual header_str
handling.

There's a case where this can start making things worse, rather than better.
If you know that templates in files are treated as bytes, you might be passing
in strings pre-encoded into UTF-8.  If that was the case, it will now become
mojibake.

Finally, plugins that read kit contents for uses as text will need upgrading.
The only one I know of like this is my own
Email::MIME::Kit::Assembler::Markdown.  I will fix it.  The trick is: look at
what content-type is being built and consider using get_decoded_kit_entry
instead of get_kit_entry.

I think this is an important change, and worth the breakage.  Please look at
your use of EMK and test with v3.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

* p...@cpan.org [2016-05-23T13:05:39]
> I created new perl module Email::Address::XS for parsing and formatting
> email groups or addresses. Parser is borrowed from dovecot and that part
> implemented in C/XS.

Cool!

> Thanks to named group support I would like to extend Email::MIME module
> to allow passing directly Email::Address::XS objects, not only string
> headers to make MIME encoding and decoding from applications easier.
> 
> What do you think about it?

I'm not sure what you're suggesting.  Do you mean:

  Email::MIME->create(..., header => [ To => $addr_xs, ...  ]);

...as opposed to:

  Email::MIME->create(..., header => [ To => $addr_xs->as_string, ...  ]);

?  Could you elaborate?

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

* p...@cpan.org [2016-05-28T16:48:40]
> Basically yes. From caller perspective I want to pass email address 
> object and let Email::MIME to do MIME encoding correctly. Something like 
> this:
> 
> my $email = Email::MIME->create(
>   header_addr => [ ... ],
> );

I think that requiring people to break headers up even further into to add
a "header_addr" argument is a bit much.  And why header_grps?  How about object
that represents the group?  Then the existing header and header_str arguments
can start silently accepting these objects and doing the right thing.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

* p...@cpan.org [2016-07-12T11:43:02]
> On Monday 04 July 2016 01:52:41 Ricardo Signes wrote:
> > 
> > I'd stick to header_str, I think, but I'm not sure.  At any rate:
> > yes.
> 
> And this is what I do not like... to pass objects to function with name 
> header_str. That name sounds like it takes string, not object (or more 
> objects)...

Either we can add a new name, so people end up having to give "header_str" and
"header_obj" or we can say "in general everything uses header_str, which
follows these simple rules."  I would rather do that.

> > > Still do not know how to handle non-MIME headers correctly in
> > > Email::MIME module. We can either create blacklist of non-MIME
> > > headers and extend it every time when somebody report problem or
> > > create whitelist of MIME headers... Or let caller to decide if his
> > > header must be MIME-encoded or not.
> > 
> > I'm sorry, I don't understand.  Could you elaborate?
> 
> If passed pair (header-name, header-value) needs to be MIME encoded or 
> not. Currently there is blacklist in Email::MIME for header names which 
> are never MIME encoded (like Message-Id, Date, ...) when passing as 
> header_str.

So, I'd assume we'd go forward with:

* if you know exactly octets you, the user, want in the header field, use
  "header", but this is likely rare
* if you want to provide a string for a field that's pretty much just a string,
  use header_str and if it requires special handling, we do our best, which
  should get better over time
* but if things are complicated, use an object that represents the structured
  data

I don't like the idea that this will be broken further by adding the object
behavior, though.

  $email->header_str_set($field => $email->header($field));

...should not break things.

> > "header_str" is "text string" which means it will get encoded.
> 
> Not exactly, there are exceptions (Message-Id, Date, ...) plus special 
> behaviour for addresses headers.

Those /mostly/ still get encoded, but we know that the strings are meant to be
structured, so we try to deconstruct them and encode them correctly.
I think those fields that get passed through unchanged are probably in error at
least insofar as they let you put non-7-bit-clean data in your headers.  This
should probably be fatal:

  header_str => [ Date => "\N{SMILING FACE WITH HORNS}" ]

> Addresses and groups are really something different as previous types 
> (strings). And if we threat them as objects, I would rather see e.g. 
> header_obj (or other different name) instead mixing it again with 
> header_str (which already have exceptions :-(). This is my initial 
> reason for header_addr/grps to distinguish it.

My feeling is that Perl programmers are used to polymorphic interfaces, and
that multiplying the number of ways to specify headers is a needless confusion.
What is the benefit to the end user of splitting things up?

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

My coworkers have returned to the other side of the world!  I attended YAPC!  i
had a vacation!  I am back.

* p...@cpan.org [2016-06-01T12:44:01]
> On Tuesday 31 May 2016 02:42:48 Ricardo Signes wrote:
> > * p...@cpan.org [2016-05-28T16:48:40]
> > 
> > > Basically yes. From caller perspective I want to pass email address
> > > object and let Email::MIME to do MIME encoding correctly. Something
> > > like this:
> > > 
> > > my $email = Email::MIME->create(
> > > 
> > >   header_addr => [ ... ],
> > > 
> > > );
> > 
> > I think that requiring people to break headers up even further into
> > to add a "header_addr" argument is a bit much.  And why header_grps?

So, you had some responses to this which were quite helpful.

My suggestion was meant to be something like "why not make Email::MIME
understand some kind of object as the value in a header?"  I think this is
still right.

Your main responses were (please correct me if I am misunderstanding them):

1.  it should be possible and easy to supply a list of address objects
2.  it should be possible to have a named group, but not required
3.  we don't want ambiguity in how objects passed to (header_str => [...])
are interpreted

What if we defined a role (here, just a well-known name) called
Email::MIME::Header::Value, which is used to signal that a particular method,
say "as_mime_header", should be used to stringify?

When building the header, the code will do something like:

  $string = $name . ": "
  . ($value->DOES('Email::MIME::Header::Value')
  ? $value->as_mime_header
  : "$value");

No existing object will become confused by this change, only objects which do
the new role.

Then Email::Address::XS could provide some helper routines, so you could write
and of:

    From => 'r...@cpan.org'

From => Email::Address::XS->new(...)

From => address('r...@cpan.org', 'Ricardo SIGNES')

From => addrlist( address('r...@cpan.org', 'Ricardo SIGNES'), ... )

From => addrgroup( Humans => address('r...@cpan.org', 'Rik'), ... )

It might be best to make the first code sample actually do:

  $string = $name . ": "
  . ($value->DOES('Email::MIME::Header::Value')
  ? $value->as_mime_header($name, $mycrlf)  # <-- changed
  : "$value");

...to let the object do folding.  I'm not sure about that one.  I'd want to
double-check whether there's a reason to not always do the folding of the
post-stringified form in Email::MIME.

Anyway, this avoids adding multiple more places to set headers and makes the
API extensible for other header types like Message-ID, etc, in the future.

What do you think of this all?

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

* p...@cpan.org [2016-07-03T08:39:22]
> On Friday 01 July 2016 02:51:31 Ricardo Signes wrote:
> 
> > What if we defined a role (here, just a well-known name) called
> > Email::MIME::Header::Value, which is used to signal that a particular
> > method, say "as_mime_header", should be used to stringify?
> 
> In this case, do we need role at all? Is not existence of method 
> "as_mime_header" enough?

That method alone is fine with me.

> And all this would be passed via header or header_str?

I'd stick to header_str, I think, but I'm not sure.  At any rate: yes.

> If address(), addrlist() and addrgroup() returns those objects (with 
> as_mime_header() method) it could be usable...
> 
> But I was thinking about using same syntax in Email::MIME for passing 
> addrlist/addrgroup as is in Email::Address::XS format_email_addresses 
> and format_email_groups functions.

I'm afraid I don't understand what you mean.

> In my opinion folding and unfolding should be done in Email::Simple. I'm 
> not huge fan of adding folding and CRLF code into Email::Address::XS as 
> it has nothing to do with it. That module parse and format one line of 
> list of addresses.

I agree.  I think if we start with the API described above, and leave the
folding for the message to perform, we'll be okay.  We can certainly find out
by writing the code!

> > What do you think of this all?
> 
> Still do not know how to handle non-MIME headers correctly in 
> Email::MIME module. We can either create blacklist of non-MIME headers 
> and extend it every time when somebody report problem or create 
> whitelist of MIME headers... Or let caller to decide if his header must 
> be MIME-encoded or not.

I'm sorry, I don't understand.  Could you elaborate?

> Basically we need unambiguous way to specify:
> 
> * ascii string which will never be MIME-encoded (error for unicode char)
> * unicode string which will be MIME-encoded if contains unicode char
> * addresses/groups - but again with ability to specify if do MIME-encode

We have that, right?

"header" is "already encoded", which is another way of saying "do not encode
this."

"header_str" is "text string" which means it will get encoded.

The address or groups thing falls under "object."  I had assumed it would,
itself, know how to become MIME encoded.  This is important, because the
semantics of what gets encoded differ per field type.  So, as_mime_header is
the encoded form.  If you want to offer an unencoded form, as_string seems like
the obvious method.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

* p...@cpan.org [2016-08-20T06:01:16]
> Email::MIME is module which automatically do any MIME encoding/decoding 
> without user interaction, so that decoding must be done automatically 
> and without such "upgrade" function.

So do you mean that whenever someone reads the header with a specific method,
the header is parsed just-in-time?

If so, this seems like something very easy to add in an Email::MIME subclass to
show it off.

Of course, it's also easy to take the hypothetical code behind
"upgrade_headers($email)" to do something just like this, on a per-known-header
basis.

> I do not want to do that decode manually or call some "upgrade" function 
> which you propose... Reading one email header should not change internal 
> email structure.

I never suggested that anything was changed by virtue of being read, but rather
that one could explictly upgrade structures if desired.

> Also it must be possible to get either named groups from Original-Cc 
> header or only list of addresses. And I think with your proposal API it 
> is not possible. You would need to call some "downgrade" function and 
> then "upgrade" it to another object or so...

Why would this not be possible?  There is some object storing the mailboxes
structure, and it provides methods that answer the questions one needs to ask.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Email::Address::XS

[Ricardo Signes]

* p...@cpan.org [2016-09-18T11:40:53]
> Currently passing string values of From/To/Cc/Bcc/... headers into 
> header_str() method is broken in Email::MIME. That is because 
> Email::MIME currently uses Email::Address for generating those header 
> values (which is broken) and then MIME encode those broken outputs.
> 
> Email::Address::XS has (looks like) correctly implemented formatter and 
> so it is needed to correctly MIME encode From/To/Cc/Bcc headers.

I suggest making Email::MIME use Email::Address::XS if it is available, and
adding Email::Address::XS to the recommended prereqs of Email::MIME.
The right behavior will be easy to get, and usually be installed, but it will
be possible to proceed with less correct behavior if you haven't got a compiler
(for some sad reason).

Part of the question is: how wrong do things go, in what circumstances, if
Email::Address is substituted for Email::Address::XS.

> As compromise could be: Whole Email::MIME will not depends on module 
> Email::Address::XS. But if somebody want to pass Unicode string (via 
> header_str) to Email::MIME then MIME encoding will be done via 
> Email::MIME::Header::AddressList (which will use Email::Address::XS). So 
> if caller encodes manually From/To/Cc/... headers and pass them via 
> header_raw() then Email::Address::XS will not be needed.

Specifically, I think, a non-ASCII string.  I'm guessing that most/many users
are really just passing in fixed ASCII strings, so this rule wouldn't affect
them at all.  Users passing in non-ASCII would start getting a "automatic
encoding of non-ASCII $field header requires " error.  Seems okay.

> And can be Email::MIME::Header::AddressList part of Email-MIME 
> distribution (even if only this module will depends on XS)?

I guess so.  We need to mark this stuff experimental for a while, I think, too.

-- 
rjbs


signature.asc
Description: Digital signature

Re: the state of art of CPAN

[Ricardo Signes]

* Marc Chantreux <m...@unistra.fr> [2017-03-06T03:32:48]
> On Sat, Mar 04, 2017 at 07:28:19PM -0500, Ricardo Signes wrote:
> > Cool, good luck!
> 
> people are really enthousiatic and i really would like to start thinking
> about a "PSGI for SMTP".

What do you think this would look like?  Email::Sender is an abstraction layer
over sending/delivering, and one could write multiple SMTP-backed transports
for it, if needed.

> > For other sending, I use Email::Sender and Email::Sender::Simple.  For
> > faster, indexed mail using JMAP, I use Cyrus, via our custom internal stack
> > at FastMail.
> 
> awesome you let us know as we were in discussion about it: is it worth
> to test cyrus as a distributed storage for the archive workers.

I can't speak to Cyrus's performance as a distributed store per se, as opposed
to having your own distribution layer atop multiple Cyruses.

I can say that Cyrus is quite performant and I've found it fairly painless to
work with.  As it stands now, I am largely a consumer of its IMAP and JMAP
servers, rather than an administrator of it.  Using it to store and quickly
read through large amounts of mail has been pleasant.

> > If you have any more specific questions, I'm happy to answer.
> 
> we can go public on this mailing list about the things we are
> experimenting so everyone can share ideas? 

Sure thing.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Patches for Email::* modules

[Ricardo Signes]

* p...@cpan.org [2017-11-24T08:08:28]
> There is a big silence so I do not know if Email::* modules are going to be
> unmaintained or if there is planned some activity. Thanks!

I do plan to work through these.  The situation roughly like this:

* I am very busy
* Insufficiently-reviewed changes could make me much more busy

So, code review keeps getting bumped while I try to try to make time.  Part of
the reason I'm busy is my day job, which relies, in part, on some of these
libraries.  So, I should be able to do some of this stuff on the clock soon.  I
just haven't been because I've been trying to get some other things done,
first.

I'll schedule a few days between Christmas and February 1, if not sooner.

-- 
rjbs


signature.asc
Description: Digital signature

Re: Any advice for a searchable web archiver ?

[Ricardo Signes]

* Marc Chantreux  [2017-11-20T14:42:24]
> > We're using Xapian as part of Cyrus IMAP, and it's quite useful for
> > what we're doing,
> 
> do you think this should be enough to store mailing lists archives?

Based on personal experience, yes.

-- 
rjbs


signature.asc
Description: Digital signature