Re: Are Email::* modules unmaintained?
* p...@cpan.org [2019-01-17T06:25:04] > I'm really disappointed as you have not wrote anything about this topic > as people are already there... I'm not dead. I am avoiding this list because I find your messages and attitude about it to be really off-putting, and whenever I come back, I feel like I'd rather go do something else. My position remains: this code works, and any change needs to be well vetted, and so far I haven't spent enough time to be satisfied that it's okay to go. Maybe I'll have a go at it again soon. When you first showed up to work on Email::MIME headers code, I said "you'll probably be happier if you just fork it," but you didn't. Now you're talking about trying to take the namespace over, which won't happen as long as I'm responsive to PAUSE admins. Which I am. I am happy to work on this project some, but when I show up and see obnoxious messages about how I owe anybody anything, or how I'm a deadbeat, it's a pretty good way to get me to say, "Hey, I'm just going to say it works well enough and also now I'm going away." I'll make another pass through the PRs next week. For now, I'm going to go do something more enjoyable. -- rjbs signature.asc Description: Digital signature
Re: Are Email::* modules unmaintained?
* p...@cpan.org [2018-06-25T05:08:32] > So what is status of Email::* modules? Does silence now mean that > modules are abandoned / unmaintained? I guess so. You are mistaken. The answer is the same as every other time you have asked: I am here, I am alive, I am interested, and I am extremely busy, so these don't often make it to the top of my list. We don't have any sort of sense of community here or a practice of shared code review, so everything comes down to "when does Rik have the time to consider each change." That is "not all that often." If we had three or four people who committed to create a better sense of group responsibility, maybe we'd see more progress. The last thing you wrung your hands about "we have no maintainer!" was something for which we didn't even have a clear and isolated fix, so the task here wasn't "nobody will review and apply this commit," but "nobody is willing to write a fix." So, in that sense, yes, nobody is doing active maintenance on Email::Address much of the time, but somebody is alive. With Email::MIME, keep in mind that part of maintenance is not just applying any old patch, because people rely on this software to not change too radically. So, there has to be not only patches, but also review and consideration, so "we aren't adding new features" isn't always a sign of disrepair, but can also indicate "we don't have the time to predict the impact of these changes on existing users." So: you can wait for me to have more time, or people with some vested interest can agree to step up to do more in general. I'm not going to be shamed into doing a release that might break things and cause me ten times as much work as leaving things be. -- rjbs signature.asc Description: Digital signature
Re: Any advice for a searchable web archiver ?
* Marc Chantreux [2017-11-20T14:42:24] > > We're using Xapian as part of Cyrus IMAP, and it's quite useful for > > what we're doing, > > do you think this should be enough to store mailing lists archives? Based on personal experience, yes. -- rjbs signature.asc Description: Digital signature
Re: Patches for Email::* modules
* p...@cpan.org [2017-11-24T08:08:28] > There is a big silence so I do not know if Email::* modules are going to be > unmaintained or if there is planned some activity. Thanks! I do plan to work through these. The situation roughly like this: * I am very busy * Insufficiently-reviewed changes could make me much more busy So, code review keeps getting bumped while I try to try to make time. Part of the reason I'm busy is my day job, which relies, in part, on some of these libraries. So, I should be able to do some of this stuff on the clock soon. I just haven't been because I've been trying to get some other things done, first. I'll schedule a few days between Christmas and February 1, if not sooner. -- rjbs signature.asc Description: Digital signature
Re: the state of art of CPAN
* Marc Chantreux [2017-03-06T03:32:48] > On Sat, Mar 04, 2017 at 07:28:19PM -0500, Ricardo Signes wrote: > > Cool, good luck! > > people are really enthousiatic and i really would like to start thinking > about a "PSGI for SMTP". What do you think this would look like? Email::Sender is an abstraction layer over sending/delivering, and one could write multiple SMTP-backed transports for it, if needed. > > For other sending, I use Email::Sender and Email::Sender::Simple. For > > faster, indexed mail using JMAP, I use Cyrus, via our custom internal stack > > at FastMail. > > awesome you let us know as we were in discussion about it: is it worth > to test cyrus as a distributed storage for the archive workers. I can't speak to Cyrus's performance as a distributed store per se, as opposed to having your own distribution layer atop multiple Cyruses. I can say that Cyrus is quite performant and I've found it fairly painless to work with. As it stands now, I am largely a consumer of its IMAP and JMAP servers, rather than an administrator of it. Using it to store and quickly read through large amounts of mail has been pleasant. > > If you have any more specific questions, I'm happy to answer. > > we can go public on this mailing list about the things we are > experimenting so everyone can share ideas? Sure thing. -- rjbs signature.asc Description: Digital signature
Re: the state of art of CPAN
* Marc Chantreux [2017-02-21T09:53:45] > hello people, Hi, Marc! Sorry for the delayed reply. I'm afraid I'm often a bit behind on these things, but I am here and alive and interested! > About sympa, we want to remove a lot of homebrew code by replacing them > with existing CPAN modules or tools so there are discutions on the > #sympa@freenode channel about the Email part. Cool, good luck! > There are 3 parts about it: > > * recieve, transform and send emails back > * archive them in maildir (mbox?) format > * produce an elastic, indexed view of those mail(box|dir) (maybe using > jmap protocol?) I tend to represent messages as Email::MIME or MIME::Entity. Both have their flaws. For writing mail to a Maildir, I use Email::Sender::Transport::Maildir. For other sending, I use Email::Sender and Email::Sender::Simple. For faster, indexed mail using JMAP, I use Cyrus, via our custom internal stack at FastMail. If you have any more specific questions, I'm happy to answer. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2017-01-14T15:32:57] > So lets move. This is implemented in my pull request: > https://github.com/rjbs/Email-MIME/pull/35 Done! -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-09-18T11:40:53] > Currently passing string values of From/To/Cc/Bcc/... headers into > header_str() method is broken in Email::MIME. That is because > Email::MIME currently uses Email::Address for generating those header > values (which is broken) and then MIME encode those broken outputs. > > Email::Address::XS has (looks like) correctly implemented formatter and > so it is needed to correctly MIME encode From/To/Cc/Bcc headers. I suggest making Email::MIME use Email::Address::XS if it is available, and adding Email::Address::XS to the recommended prereqs of Email::MIME. The right behavior will be easy to get, and usually be installed, but it will be possible to proceed with less correct behavior if you haven't got a compiler (for some sad reason). Part of the question is: how wrong do things go, in what circumstances, if Email::Address is substituted for Email::Address::XS. > As compromise could be: Whole Email::MIME will not depends on module > Email::Address::XS. But if somebody want to pass Unicode string (via > header_str) to Email::MIME then MIME encoding will be done via > Email::MIME::Header::AddressList (which will use Email::Address::XS). So > if caller encodes manually From/To/Cc/... headers and pass them via > header_raw() then Email::Address::XS will not be needed. Specifically, I think, a non-ASCII string. I'm guessing that most/many users are really just passing in fixed ASCII strings, so this rule wouldn't affect them at all. Users passing in non-ASCII would start getting a "automatic encoding of non-ASCII $field header requires " error. Seems okay. > And can be Email::MIME::Header::AddressList part of Email-MIME > distribution (even if only this module will depends on XS)? I guess so. We need to mark this stuff experimental for a while, I think, too. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-09-17T19:05:51] > $class->from_mime_string() will take raw MIME encoded string and returns > new object of $class (which will have decoded string parts) > $object->as_mime_string() will convert (Unicode) $object into raw MIME > encoded string > > It is OK for you? That all sounded fine. I think the paragraph I left quoted overspecifies a bit. Whether the object is storing things decoded or not isn't any of our concern as long as it has those two methods. But I think we're on the same page. The Email::Address::XS use should be optional, as right now people can install Email::MIME in an compiler-free environment. We can add it as a recommended prereq. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-09-12T03:26:52] > And as I wrote if Email::MIME is not good place, then what about other > modules like Email::MIME::Header::Address (or invent other name) which > will use Address parse/format functions and will also do that MIME > encode/decode procedure? We can maybe add classes also for other headers > (like you suggested for DKIM signatures, etc...). I had started to write a lot of reply on the previous parts of your email, but I think that this is the only part that really matters in the end. Yes, I think some thing like that is sufficient. In the end, I think what's best is: * a thing that can take a raw (encoded) header string and give you an object * ...which is an object with access to the header's structured data * ...which you can turn back into a raw header to store as needed With that facility, people can plug in (header => class) configuration and things just go. We can start off suggesting, for example, an address one. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-09-05T04:25:11] > I do not want to add ->as_mime_header (or other function) which will do > MIME encoding/decoding into Email::Address::XS. That module is for > formating and parsing email addresses headers, not for MIME > encoding/decoding. Same as it is for Email::Address module. The best way to know how to properly encode a structured header field is to know both its structure and the way that the structure ie meant to be encoded. For example, to know that a `mailboxes` structure may have a display-name, which would be words, which can be encoded, and may have an addr-spec, which is not words, and so cannot be encoded. Parsing and encoding are not separable concerns, here, because to know whether to decode a part, one most know what part it is, which means it has been parsed. You can only properly decode the structured data by knowing the relationship between structure and encoding. Then, you can only encode the data by knowing the same. This means that any interface between Email::MIME and some structured field representation has a much more complex API, if Email::MIME is responsible for the encoding and decoding. It doesn't just need a map of field name to class name, but also instructions on how structured data are encoded and decoded. This seems like it becomes a nasty mess of deep coupling. Am I making some fundamental mistake, here? Anyway, isn't it certain that people who parse addresses from headers will want to get a decoded form? Surely this will be common: my @recipients = map {; $_->phrase_str // $_->address } Address->parse( $email->header('To') ); The Dovecot parser (as I recall) does not decode encoded-words, so phrase_str is easy to write. Does every consumer of Address need to know how to decode? Further, won't people want to write: my $to = Address->new("김정은", "k...@example.biz"); ...and then pass that object on to something that knows what to do with it? Does every possible consumer of Address need to know how to encode an Address object? -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
I know I'm taking a long time between replies. Thanks for being patient. I've been rotating through "out of town" and "catching up with work backlog from being out of town," basically, and in the leftover time, I don't have any brain left for anything much at all. This week is another "all work all week" week, but maybe the week after things will even out for a while. * p...@cpan.org [2016-08-25T03:40:20] > On Wednesday 24 August 2016 22:55:05 Ricardo Signes wrote: > > > > I don't understand "you have no idea about arbitrary object." Obviously you > > would get a type of object based on the header in question. > > Then you need to create mapping from header name to object name. Plus > this does not solve problems for extended/application specific header > (X-Something) which can be used for type which application wants. Yes, you need that mapping, and you extend it on an application-specific basis. > > This reads like, "Look, just use the API that you don't like because I > > already > > It is not like it... I apologize, this was an impolite response. > I would rather know what is wrong with it? And which part? Both > Email::Simple & Email::MIME? Or only some subpart of it? And both > getting and setting headers? Or only getting them? What I meant was: we were talking about questions of API design, and you moved to implementation, which I think is premature. > Do not take me wrong, but to check that API is usable, you need to > implement at least some POC and try to use it yourself. If it does not > meet everything needed, then you need to rework it. And this is what now > did. I agree that you need to test an API to determine whether it is sufficient, but it's also possible to see something is insufficient before trying. Since I don't think the header_addrlist API is sufficient, it seems like implementation is jumping the gun, to me. > Look at my proposal just for first version and lets change parts which > are not OK for you. I do not believe that everything is totally wrong. Okay! First, I think we should just leave Email::Simple alone. In general, I think the cases for using Email::Simple are very few, and almost nobody should ever use it. Giving it new and ostensibly MIME-related features seems unnecessary. Having said that, I'm not going to look at the Email::Simple changes in depth. (We definitely don't want to make installing Email::Simple require loading Email::Address::List::XS, I'll note.) I think that ->format is probably not a great name choice, as it might exist other places too easily. For example, Email::Address has a ->format, but I don't think it will be suitable for this, as it doesn't encode properly. This is why I originally suggested something almost guaranteed not to clash, like ->as_mime_header. We can assume that programmers won't have to call this very often, only the innards of Email::MIME, so it's okay if it's a bit wordy. The Email::MIME changes look like they could be broken up into several PRs, some of which would be obviously good to apply immediately, like removals of dead code and pointers to bad modules. Primarily, I don't like the special weight given to the addrlist header. While it's likely to be the most common one, I think that implementing it as a special case rather than an application of the general case, is going to lead to problems. (Just yesterday I spent much of the day on DKIM, and it was clear that Authentication-Results and Domain-Signature could both usefully have special objects.) > [...] > So easy extensible API needs to have one method which do that. Now I > have only idea with something like this: > > my $addrlist = $email->header_to_obj("Cc", "Email::Address::List::XS"); > > That will convert header "Cc" to object Email::Address::List::XS and > MIME decode parts which needs to be decoded. > > (Maybe class name could be optional and some mapping table for most > common headers could be prepared) I think this is all plausible. The parts that are important to me are: * objects working for all headers equally well * a registry of common field-name-to-class-name mappings > That method still needs to be know how to MIME decode object > Email::Address::List::XS... I'm not sure what you mean, here. Do you mean that if we've stored a header entry as an object that has an as-mime-encoded-string method, we also end up needed a means to get it as-decoded-string? I'm afraid I just don't understand the sentence. Your changes to Email::Simple don't store objects, but produce them on demand. I'm thinking of: https://github.com/rjbs/Email-Simple/compare/master...pali:master#diff-8816e211b9069c6bf
Re: Email::Address::XS
* p...@cpan.org [2016-08-23T03:56:24] > > > Also it must be possible to get either named groups from Original-Cc > > > header or only list of addresses. And I think with your proposal API it > > > is not possible. You would need to call some "downgrade" function and > > > then "upgrade" it to another object or so... > > > > Why would this not be possible? There is some object storing the mailboxes > > structure, and it provides methods that answer the questions one needs to > > ask. > > Because you have no idea about arbitrary object. If you want to e.g. > decode Email::Address::XS object, you must decode only ->phrase() and > ->comment() parts! Not others. I don't understand "you have no idea about arbitrary object." Obviously you would get a type of object based on the header in question. > Anyway, lets move forward. I already implemented something and send > information in email with subject "Email::Simple & Email::MIME with > Email::Address::XS" to pep mailing list... > > I think this is good approach to provide usable API + ability to extend > code for other objects... This reads like, "Look, just use the API that you don't like because I already wrote some code." That's not going to sway me. What happens when someone wants a Date object for the Date header? Do we add header_date? Then header_rcvd for Received headers, and so on? This interface leads to either a proliferation of these things or to some line where we say, "well *these* headers are important enough and *these* are not." On the other hand, a generic mechanism is generic. You can always publish your work as a subclass, if you think it that popular acclaim will convince me I'm wrong. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-08-23T03:50:03] > That is really bad API :-( This is the *low level* API which of course the user does not call. They'd want the version of Email::MIME that gives them these header objects on the fly. I suggested there'd be a means to "upgrade" all the known headers so that ->header_thing('To') would give you an object. If that worked, it would probably be useful to have an analog to ->new that did this while initializing. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-08-20T06:01:16] > Email::MIME is module which automatically do any MIME encoding/decoding > without user interaction, so that decoding must be done automatically > and without such "upgrade" function. So do you mean that whenever someone reads the header with a specific method, the header is parsed just-in-time? If so, this seems like something very easy to add in an Email::MIME subclass to show it off. Of course, it's also easy to take the hypothetical code behind "upgrade_headers($email)" to do something just like this, on a per-known-header basis. > I do not want to do that decode manually or call some "upgrade" function > which you propose... Reading one email header should not change internal > email structure. I never suggested that anything was changed by virtue of being read, but rather that one could explictly upgrade structures if desired. > Also it must be possible to get either named groups from Original-Cc > header or only list of addresses. And I think with your proposal API it > is not possible. You would need to call some "downgrade" function and > then "upgrade" it to another object or so... Why would this not be possible? There is some object storing the mailboxes structure, and it provides methods that answer the questions one needs to ask. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-08-18T17:35:10] > On Thursday 18 August 2016 23:21:28 Ricardo Signes wrote: > > As you say, 1 and 2 are dealt with. For 3 or 4, you want to have an > > object in the header slot, rather than a string. Once you've done > > that, you use its methods. If the object's To field stores a > > string, you "upgrade" it with something like: > > > > $email->header(To => mailbox_headers_from( $email->header('To') ); > > > > ...and it seems like one would quickly amass some sort of routine > > like: > > > > upgrade_headers($email); > > > > ...that would upgrade all the headers it knows about. > > Can you describe (or write code) how you imagine that I get header > "Original-Cc" in form of addresses in list of named groups from email > which is stored in string scalar? I'm not sure that I understand how you > mean it... Here's a verbose form: # Get an email. my $email = get_some_email_mime(); # Get the header -- the (unfolded) raw bytes. my $cc_hdr = $email->header_raw('Original-CC'); # parse it into an object my $cc_obj = parse_mailboxes( $cc_obj ); # put that object into the header: $email->header_set('Original-CC', $cc_obj); # get the raw mime-encoded bytes again: my $cc_hdr2 = $email->header_raw('Original-CC'); # get a list of sub-object from the object's imaginary interface: my @boxes = $email->header_obj('Original-CC')->boxes; -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-08-08T17:41:04] > Here we are need to deal with objects which internally needs to be MIME > encoded and objects which mustn't. I don't think this matters. If an object is passed in, it must be able to produce a MIME encoded form. Even if you say: header_str => [ header => $obj_based_on_text ] ...the object will be required to have an as_mime_header method, which must produce an encoded form. The point was to make the behavior of objects unambiguous no matter where you put them, not to multiply the possible semantics of objects by two. The only difference between header and header_str, I said, would be how they treated plain strings. > If I create Email::MIME object from input string, I would like to get: > > 1) Raw (ASCII) string representation of To: field > > 2) Unicode string representation of To: field > > 3) List of Email::Address::XS objects which are in To: field > > 4) List of named groups with Email::Address::XS objects of To: field > > For 1) and 2) I can use ->header_raw and ->header_str methods. For 3) > and 4) are needed new method(s). Ideally if caller is able to get > original MIME encoded objects (where in ->phrase part of address object > is still MIME encoded) and also if objects strings are Unicode. As you say, 1 and 2 are dealt with. For 3 or 4, you want to have an object in the header slot, rather than a string. Once you've done that, you use its methods. If the object's To field stores a string, you "upgrade" it with something like: $email->header(To => mailbox_headers_from( $email->header('To') ); ...and it seems like one would quickly amass some sort of routine like: upgrade_headers($email); ...that would upgrade all the headers it knows about. > I can accept that both "header" and "header_str" will work with objects, > but I think that my suggestion about do not encoding "phrase" string > part of object passed to "header" is useful... I agree that if you provide an object, Email::MIME should not try to further encode anything, and that it should trust the object to provide its own encoded form. (Email::MIME will line-fold, though, as discussed.) -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-08-02T17:03:07] > I can imagine, that people could be confused about header_str meaning. > It has suffix _str and I would expect it needs (Unicode) string, not > object... Name "header" is better as it does not say it needs string. People will want to be able to pass non-ASCII strings in as subject, meaning that header is not suitable for the "one true list of fields." Passing in a pre-encoded value is pretty sure to be the exception, not the rule. In other words, I think this would be more sensible: header_str => [ Foo => raw_mime($header_raw), Bar => "Text string to be encoded", Baz => $message_id_object, ], The alternative, using header, is: header => [ Foo => $header_raw, Bar => mime_encode("Text string to be encoded"), Baz => $message_id_object, ], Of course, there's no reason that both header and header_str can't accept these objects, and the user can pick whichever is more convenient, right? The difference between header and header_str becomes only the behavior for plain strings. > > * if you know exactly octets you, the user, want in the header field, > > use "header", but this is likely rare > > Do you mean $email->header_raw_set()? > > I think it is not rare to encode header (to MIME) externally and then > pass ASCII 7bit string to $email. At least I see this usage for From > header (in previous version of Email::MIME encoding of From/To/Cc > headers was totally broken). I mean both "header" in the initializer and header_set and header_raw_set, which are equivalent. > > unchanged are probably in error at least insofar as they let you put > > non-7-bit-clean data in your headers. This should probably be > > fatal: > > > > header_str => [ Date => "\N{SMILING FACE WITH HORNS}" ] > > Here is problem: Should Email::MIME understand meaning of email headers? I think its level of understanding is roughly appropriate, although imperfect. It's meant to prevent you passing in a string of addresses that are naively correct but actually need encoding. It's better if people use something structured for headers where this is complex, though. > Here we see that header_str does not say (or specify) which string must > be specified as parameter. Unicode string? Arbitrary 8bit string? 7bit > ASCII string? Or ASCII subset visible characters? It says, in the docs for create: This method creates a new MIME part. The "header_str" parameter is a list of headers pairs to include in the message. The value for each pair is expected to be a text string that will be MIME-encoded as needed. A similar "header" parameter can be provided in addition to or instead of "header_str". Its values will be used verbatim. *text string*, not byte string. > I think we should unify API for it. And ideally describe into > documentation how to correctly use it. Agreed. > That /mostly/ with special exceptions for Message-Id or Date is wrong. I don't think I agree. I think that the behaviors on address list headers is useful. Ideally, people use methods to produce objects for structured headers. email_addr_list() for example. The current behavior is roughly to saying: bare strings for these headers are implicitly parsed into objects that then encode things. That's roughly how the message list headers are implemented. That the Date field is bogus is unfortunate. I imagine that really there are only about 3 things to worry about: * mailbox and mailbox list * fields that do not allow encoded words (and so must be 7-bit clean) * fields that are sequences of words If people know how to produce the already-encoded form, they can do so already. If they don't, but know what the decoded string would look like, the current system can continue to improve over time. In other words: if you say "I have this structured data and it isn't yet encoded, please encode it for me," we need to understand it exactly enough to know how to encode it, so this behavior is necessary if header_str is going to work for structured fields. > 1) Function name say what it accept I am not very swayed by this. Users can be surprised once for a brief moment when they see [ header_str => [ From => $object ] ] and then they know forever. On the other hand, having multiple sets of headers to write is annoying every time. > 2) No problem with meaning which type of string is accepted (subset > ASCII, ASCII or Unicode as described above) This is already unambiguous. _str forms always expect character strings. > 3) Possible performance optimization (less objects are created) How? > And there is another problem still not solved. From $email object it is > needed also to read From/To/Cc headers and user (caller) of Email::MIME > module is sometimes interested in de-composited addresses objects (e.g. > when want to parse each email address in CC field) and sometimes > interested only in one string representation (e.g. want to write header > to STDOUT)
Re: Email::Address::XS
* p...@cpan.org [2016-07-12T11:43:02] > On Monday 04 July 2016 01:52:41 Ricardo Signes wrote: > > > > I'd stick to header_str, I think, but I'm not sure. At any rate: > > yes. > > And this is what I do not like... to pass objects to function with name > header_str. That name sounds like it takes string, not object (or more > objects)... Either we can add a new name, so people end up having to give "header_str" and "header_obj" or we can say "in general everything uses header_str, which follows these simple rules." I would rather do that. > > > Still do not know how to handle non-MIME headers correctly in > > > Email::MIME module. We can either create blacklist of non-MIME > > > headers and extend it every time when somebody report problem or > > > create whitelist of MIME headers... Or let caller to decide if his > > > header must be MIME-encoded or not. > > > > I'm sorry, I don't understand. Could you elaborate? > > If passed pair (header-name, header-value) needs to be MIME encoded or > not. Currently there is blacklist in Email::MIME for header names which > are never MIME encoded (like Message-Id, Date, ...) when passing as > header_str. So, I'd assume we'd go forward with: * if you know exactly octets you, the user, want in the header field, use "header", but this is likely rare * if you want to provide a string for a field that's pretty much just a string, use header_str and if it requires special handling, we do our best, which should get better over time * but if things are complicated, use an object that represents the structured data I don't like the idea that this will be broken further by adding the object behavior, though. $email->header_str_set($field => $email->header($field)); ...should not break things. > > "header_str" is "text string" which means it will get encoded. > > Not exactly, there are exceptions (Message-Id, Date, ...) plus special > behaviour for addresses headers. Those /mostly/ still get encoded, but we know that the strings are meant to be structured, so we try to deconstruct them and encode them correctly. I think those fields that get passed through unchanged are probably in error at least insofar as they let you put non-7-bit-clean data in your headers. This should probably be fatal: header_str => [ Date => "\N{SMILING FACE WITH HORNS}" ] > Addresses and groups are really something different as previous types > (strings). And if we threat them as objects, I would rather see e.g. > header_obj (or other different name) instead mixing it again with > header_str (which already have exceptions :-(). This is my initial > reason for header_addr/grps to distinguish it. My feeling is that Perl programmers are used to polymorphic interfaces, and that multiplying the number of ways to specify headers is a needless confusion. What is the benefit to the end user of splitting things up? -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-07-03T08:39:22] > On Friday 01 July 2016 02:51:31 Ricardo Signes wrote: > > > What if we defined a role (here, just a well-known name) called > > Email::MIME::Header::Value, which is used to signal that a particular > > method, say "as_mime_header", should be used to stringify? > > In this case, do we need role at all? Is not existence of method > "as_mime_header" enough? That method alone is fine with me. > And all this would be passed via header or header_str? I'd stick to header_str, I think, but I'm not sure. At any rate: yes. > If address(), addrlist() and addrgroup() returns those objects (with > as_mime_header() method) it could be usable... > > But I was thinking about using same syntax in Email::MIME for passing > addrlist/addrgroup as is in Email::Address::XS format_email_addresses > and format_email_groups functions. I'm afraid I don't understand what you mean. > In my opinion folding and unfolding should be done in Email::Simple. I'm > not huge fan of adding folding and CRLF code into Email::Address::XS as > it has nothing to do with it. That module parse and format one line of > list of addresses. I agree. I think if we start with the API described above, and leave the folding for the message to perform, we'll be okay. We can certainly find out by writing the code! > > What do you think of this all? > > Still do not know how to handle non-MIME headers correctly in > Email::MIME module. We can either create blacklist of non-MIME headers > and extend it every time when somebody report problem or create > whitelist of MIME headers... Or let caller to decide if his header must > be MIME-encoded or not. I'm sorry, I don't understand. Could you elaborate? > Basically we need unambiguous way to specify: > > * ascii string which will never be MIME-encoded (error for unicode char) > * unicode string which will be MIME-encoded if contains unicode char > * addresses/groups - but again with ability to specify if do MIME-encode We have that, right? "header" is "already encoded", which is another way of saying "do not encode this." "header_str" is "text string" which means it will get encoded. The address or groups thing falls under "object." I had assumed it would, itself, know how to become MIME encoded. This is important, because the semantics of what gets encoded differ per field type. So, as_mime_header is the encoded form. If you want to offer an unencoded form, as_string seems like the obvious method. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
My coworkers have returned to the other side of the world! I attended YAPC! i had a vacation! I am back. * p...@cpan.org [2016-06-01T12:44:01] > On Tuesday 31 May 2016 02:42:48 Ricardo Signes wrote: > > * p...@cpan.org [2016-05-28T16:48:40] > > > > > Basically yes. From caller perspective I want to pass email address > > > object and let Email::MIME to do MIME encoding correctly. Something > > > like this: > > > > > > my $email = Email::MIME->create( > > > > > > header_addr => [ ... ], > > > > > > ); > > > > I think that requiring people to break headers up even further into > > to add a "header_addr" argument is a bit much. And why header_grps? So, you had some responses to this which were quite helpful. My suggestion was meant to be something like "why not make Email::MIME understand some kind of object as the value in a header?" I think this is still right. Your main responses were (please correct me if I am misunderstanding them): 1. it should be possible and easy to supply a list of address objects 2. it should be possible to have a named group, but not required 3. we don't want ambiguity in how objects passed to (header_str => [...]) are interpreted What if we defined a role (here, just a well-known name) called Email::MIME::Header::Value, which is used to signal that a particular method, say "as_mime_header", should be used to stringify? When building the header, the code will do something like: $string = $name . ": " . ($value->DOES('Email::MIME::Header::Value') ? $value->as_mime_header : "$value"); No existing object will become confused by this change, only objects which do the new role. Then Email::Address::XS could provide some helper routines, so you could write and of: From => 'r...@cpan.org' From => Email::Address::XS->new(...) From => address('r...@cpan.org', 'Ricardo SIGNES') From => addrlist( address('r...@cpan.org', 'Ricardo SIGNES'), ... ) From => addrgroup( Humans => address('r...@cpan.org', 'Rik'), ... ) It might be best to make the first code sample actually do: $string = $name . ": " . ($value->DOES('Email::MIME::Header::Value') ? $value->as_mime_header($name, $mycrlf) # <-- changed : "$value"); ...to let the object do folding. I'm not sure about that one. I'd want to double-check whether there's a reason to not always do the folding of the post-stringified form in Email::MIME. Anyway, this avoids adding multiple more places to set headers and makes the API extensible for other header types like Message-ID, etc, in the future. What do you think of this all? -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-05-28T16:48:40] > Basically yes. From caller perspective I want to pass email address > object and let Email::MIME to do MIME encoding correctly. Something like > this: > > my $email = Email::MIME->create( > header_addr => [ ... ], > ); I think that requiring people to break headers up even further into to add a "header_addr" argument is a bit much. And why header_grps? How about object that represents the group? Then the existing header and header_str arguments can start silently accepting these objects and doing the right thing. -- rjbs signature.asc Description: Digital signature
Re: Email::Address::XS
* p...@cpan.org [2016-05-23T13:05:39] > I created new perl module Email::Address::XS for parsing and formatting > email groups or addresses. Parser is borrowed from dovecot and that part > implemented in C/XS. Cool! > Thanks to named group support I would like to extend Email::MIME module > to allow passing directly Email::Address::XS objects, not only string > headers to make MIME encoding and decoding from applications easier. > > What do you think about it? I'm not sure what you're suggesting. Do you mean: Email::MIME->create(..., header => [ To => $addr_xs, ... ]); ...as opposed to: Email::MIME->create(..., header => [ To => $addr_xs->as_string, ... ]); ? Could you elaborate? -- rjbs signature.asc Description: Digital signature
Email::MIME::Kit v3
Ever since its early releases, Email::MIME::Kit had a big problem. It screwed up encodings. Specifically, imagine his manifest (I'm kinda skipping some required junk): # manifest.yaml renderer: TemplateToolkit headers: - Subject: "Message for [% name %]" alternatives: - type: text/plain path: body.txt - type: text/html path: body.html The manifest turns into a data structure before it's used, and the subject header is a text string that, later, will get encoded into MIME encoded-words on the assumption that it's all Unicode text. The files on disk are read with :raw, then filled in as-is, and trusted to already be UTF-8. If your customer's name is Распутин, strangely enough, you're okay. The header handling encodes it properly, and the wide characters (because Cyrillic codepoints are all above U+00FF) turn into UTF-8 with a warning. On the other hand, for some trouble, consider Ævar Arnfjörð Bjarmason. All those codepoints are below U+0100, so the non-ASCII ones are encoded directly, and you end up with =C6 (Æ) in your quoted-printable body instead of =C3=86 (Æ UTF-8 encoded). Now, you're probably actually okay. Your email is not correct, but email clients are good at dealing with your (read: my) stupid mistakes. If your email part says it's UTF-8 but it's actually Latin-1, mail clients will usually do the right thing. The big problem is when you've got both Ævar Arnfjörð Bjarmason and Распутин both in your email. Your body is a mish mash of Latin-1 and UTF-8 data. In Email::MIME::Kit v3, templates (or non-template bodies) loaded from disk are — if and only if they're for text/* parts — decoded into text and then, when the email is assembled, it's encoded by Email::MIME's usual header_str handling. There's a case where this can start making things worse, rather than better. If you know that templates in files are treated as bytes, you might be passing in strings pre-encoded into UTF-8. If that was the case, it will now become mojibake. Finally, plugins that read kit contents for uses as text will need upgrading. The only one I know of like this is my own Email::MIME::Kit::Assembler::Markdown. I will fix it. The trick is: look at what content-type is being built and consider using "get_decoded_kit_entry" instead of "get_kit_entry." I think this is an important change, and worth the breakage. Please look at your use of EMK and test with v3. -- rjbs signature.asc Description: Digital signature
Re: The PDF attachments sent with Email::MIME::Kit are damaged
* Octavian Rasnita [2014-11-02T08:05:54] > Please apply the following diff to Email::MIME::Kit::KitReader::Dir. Okay! I've released a new version with this patch. Enjoy! -- rjbs signature.asc Description: Digital signature
Re: things need co-maint
* Erik Logtenberg [2014-08-20T06:03:14] > Of this list of modules I use, Email-Send and MIME-Lite are on your list > of modules that need a new (co)maintainer. I urge you to consider using Email::Sender in place of Email::Send. It is a significant upgrade to testability and extensibility. > Could you incidate if these modules currently have other maintainers, > and if there are open bugs (i.e. there is already maintenance that needs > to be done). MIME-Lite had about 125 tickets when I took up comaint, and is now down to "just" 51. I am only a co-maintainer on it, but have done the only releases in the last nine years. It may be that the other maintainers would like to do more work on it, but I would be somewhat surprised. Email-Send has no open tickets. -- rjbs signature.asc Description: Digital signature
Re: things need co-maint
* Geoffrey Leach [2014-08-19T19:54:44] > Does this imply the passing of the Perl Email Project? Perhaps you ncould > elaborate. The PEP, as far as I am concerned, is a non-thing. It is a name describing nothing that exists. In about five years, the only thing that has happened associated with its name have been releases of software once under its umbrella, done by me, linking to a single static web page. If someone wants to do something with the name, great. Otherwise, nothing is being killed off that was not already moldering. -- rjbs signature.asc Description: Digital signature
Re: things need co-maint
* 'lesleyb' [2014-07-30T06:58:14] > On Tue, Jul 29, 2014 at 07:03:39AM -0400, Ricardo Signes wrote: > > There are a lot of email modules that I don't use and am not being very > > vigilant about maintaining. In some cases, I can chown them to HANDOFF on > > PAUSE and in other cases, I'm just comaint and can't do anything but declare > > that future releases from me are unlikely. > > > > These modules include: [...] > > So what would you like to see done with these modules? I want people to realize that I am not going to keep doing maintenance, and that if they need ongoing maintenance, they will need to pick up the burden or go through the other listed maintainers. -- rjbs signature.asc Description: Digital signature
things need co-maint
There are a lot of email modules that I don't use and am not being very vigilant about maintaining. In some cases, I can chown them to HANDOFF on PAUSE and in other cases, I'm just comaint and can't do anything but declare that future releases from me are unlikely. These modules include: Email-Delete Email-Folder Email-Folder-IMAP Email-Folder-IMAPS Email-Folder-POP3 Email-FolderType Email-FolderType-Net Email-LocalDelivery Email-Reply Email-Send Email-Send-IO Email-Store Email-Stuff Mail-Audit-DKIM Mail-Audit-List Mail-Audit-PGP Mail-Audit-Razor Mail-Convert-Mbox-ToEML Mail-LocalDelivery MIME-Lite -- rjbs signature.asc Description: Digital signature
Re: Derrivative code
* Matijs van Zuijlen [2014-07-13T07:17:39] > - Email::Sender is more dangerous: if through a programming error an > undefined transport is passed to sendmail(), it will happily try to send the > mail you were trying to save locally via SMTP. Don't use Email::Sender::Simple, use the Mbox (or whatever) transport directly. > - Email::Sender::Transport::Mbox will not accept messages without a sender, > whereas Email::LocalDelivery does allow this. Don't you need a sender for the From_ line? Pass an empty string as needed..? -- rjbs signature.asc Description: Digital signature
Re: Derrivative code
* Geoffrey Leach [2014-07-05T18:30:19] > I'm at work on MH.pm, which I hope to submit to CPAN as > Email::LocalDelivery:MH.pm. The code is based on > Email::LocalDelivery::Maildir.pm > > Question, in such a case is there a preference for maintaining the format, > naming conventions, etc. of the original code? I don't think it matters, as long as the interface is compatible. FWIW, I am soon going to abandon maintenance of Email::LocalDelivery. I only have one line of code still using it. For all other things, I now use Email::Sender. -- rjbs signature.asc Description: Digital signature
Re: reading webmail inbox from perl
* "Devireddy, Nagendra" [2014-01-28T08:40:18] > Imap I suggest using Mail::IMAPClient. It's easy to use and fairly well documented. -- rjbs signature.asc Description: Digital signature
Re: reading webmail inbox from perl
* "Devireddy, Nagendra" [2014-01-28T06:30:04] > Our mail server address is something like this devmail.fqdn.com and I am > trying to read inbox messages from that one by one. With what protocol? -- rjbs signature.asc Description: Digital signature
Re: Migrating mail suite from perl 5.8 32 bit to 5.12 64 bit
* Suresh Govindachar [2012-03-26T16:26:23] > Not a GLOB reference at C:/opt/perl64/site/l�ib/Net/POP3/SSL.pm line 16. I'm afraid I don't know anything about this particular problem with Net::POP3::SSL, sorry. -- rjbs signature.asc Description: Digital signature
Re: Email::Address hangs
* Joseph Crotty [2012-01-28T18:20:28] > When calling parse on a large To: header field it hangs. Large in this case > is 600 emails some with comments. I put in a length check on the To: field > to avoid this. Are there any other ways to parse long To:field fields > successfully? You could provide an anonymized version of this header, so I can look at what might be happening? Also, consider trying Mail::Address. -- rjbs signature.asc Description: Digital signature
Re: backwards incompatible asciihood change in Email::Address
* Ruslan Zakirov [2012-01-15T10:24:59] > On Sun, Jan 15, 2012 at 19:09, Ricardo Signes > wrote: > > I've updated Github's repo with a change to only reject non-ASCII in the > > email address, which really is a problem. My guess is that you were having > > a problem with the decoded phrase legally containing non-ASCII. > > Right guess. Is it legal? I don't think it is legal according to the > spec to have non-ascii phrase. As far as I recall it should be encoded > with Q/B. Sure, but people aren't (I hope) passing the mime-header-encoded content to ->parse. That will parse, but give you crap. They should be passing the decoded character string, at which point the non-ASCII phrase is legal. i.e., Email::Address parses the header's decoded character data, not its raw encoded data. I will make a release in a few hours. -- rjbs
Re: backwards incompatible asciihood change in Email::Address
* Ruslan Zakirov [2012-01-15T08:22:23] > May be it's proper from RFC point of view, but it breaks Request > Tracker application and it's going to cost a lot of development to get > it fixed. I've updated Github's repo with a change to only reject non-ASCII in the email address, which really is a problem. My guess is that you were having a problem with the decoded phrase legally containing non-ASCII. Please let me know if this helps, and if so, I will add some tests, adjust the docs, and release. -- rjbs
Re: ARF MIME reports best perl options
* Joseph Crotty [2011-11-10T23:31:39] > All, an initial patch to get the ball rolling. Untested, but hope helps. > Some sections have comments with no action taken as unsure the correct > course of action. This patch doesn't even result in compiling code. I'll have a look at what's changed in the RFC since my last release to make sure we can move to claiming version 1. Patches are definitely welcome, but it would be easiest for me if they were in a git repo or the result of git format-patch. I can muddle through other diffs; it just takes a bit longer. http://github.com/rjbs/email-arf -- rjbs
Re: ARF MIME reports best perl options
* Steve Atkins [2011-11-10T20:25:26] > > I need to be sending out ARF (i.e., RFC 5965) reports on a production > > systems, possibly 100-1,000 ARFS/second, using perl. > > That's pretty high volume. Yes. I'm not sure how Email::ARF will do, but am interested to find out... > > What's my best bet > > both in terms of performance and adherence to RFC draft spec for ARF? > > Email::ARF::Report > > look promising, but warns it's not "production ready." > > That's probably the smart choice. Email::ARF is used by me to parse oodles of reports. I have a few known bugs with generation. I'll try to close those out this week so you can give it a try. :-) -- rjbs
Re: Alternative to Mail::IMAPClient that supports proxyauth?
* Jesse Thompson [2011-06-23T15:09:24] > Does anyone know if there is an alternative to Mail::IMAPClient that > supports proxyauth. I don't know of any IMAP client worth using much beyond Mail::IMAPClient, which I use -- and with which proxy auth works. -- rjbs
Re: do I always need to specify an encoding with Email::MIME?
* James Peregrino [2011-05-02T12:53:46] > I'm trying to use Email::MIME to send a simple email with a .doc file as an > attachment. I receive it fine with Gmail, but my job email chokes on it when > it tries to scan the attachment for viruses ('UNSCANABLE'). > > send SMTP => Email::MIME->create( Please consider using Email::Sender instead of Email::Send. Seriously, it's just as easy and so much better in the long run. Ask me more on IRC if you like. > parts => [ > Email::MIME->create( > body => "The exam file is attached to this message.", > ), > Email::MIME->create( > body => io("$EXAM"), > attributes => { > filename => $EXAM_FILE_NAME, > content_type => 'application/msword', > }, > ), > ], > I suggest doing so. It won't hurt, and it can prevent various forms of error. I suggest using quoted-printable for the text and base64 for the Word document. -- rjbs
Re: pacth for MIME::Lite
* fakessh [2010-10-08T05:42:29] > hello pep network > > did you apply the patch and did you find the problem of encoding > characters in the lib MIME:: Lite > > I thank you for all your possible returns I have been very busy of late, but hope to get a bunch of outstanding patches reviewed, tested, and applied in the coming weeks. -- rjbs
Re: Fwd: Re: Email::MIME walk_parts doesn't walk all my parts
* Erik Logtenberg [2010-08-29T17:26:38] > Since this change, inspired by Ricardo Signes, fixes an obvious bug in > Email::MIME, I'd like to request this patch be pushed into git. Also a > new release of Email::MIME containing this fix would be nice. Released in 1.905, thanks! -- rjbs
Re: Email::MIME walk_parts doesn't walk all my parts
* Erik Logtenberg [2010-06-03T14:23:52] > $email->walk_parts(sub { > my ($part) = @_; > warn($part->content_type . ": " . $part->parts); > }); > > This little snippet is supposed to read an email from stdin and print > the content-types of all parts (plus the amount of subparts that it has). > > Now for most emails this works fine, but not so for attached email. For > this email it only sees the first part: This looks like an incredibly stupid bug based on the incredibly stupid ->parts method. Its behavior is really lame. I'm afraid I don't have a lot more time to look at this right now, but I'd play around with tweaking walk_parts (in Email::MIME) to use ->subparts instead, which is a lot saner. If that fixes this, it's probably a welcome patch. Thanks for the report. -- rjbs
Re: how to add a proper attachment to an email message
* fakessh [2010-05-13T16:05:31] > my question is simple > how to add a proper attachment to an email message > I use as main modules use Email::Send and use Email::Simple::Creator You need to use Email::MIME. -- rjbs
Re: Email::Folder stuck at first message
* Eddie Rowe [2010-04-14T11:41:45] > I have tried to process messages from a Mozilla Thunderbid INBOX file. Sorry it took me a month to reply. > [code] > use Email::Folder; > > my $folder = Email::Folder->new("c:/users/eddie/desktop/INBOX.txt"); > > while( my $message = $folder->next_message ) { > print join "\n", map { $_->header('Date') } $message; > } > [/code] I don't know what format Thunderbird stores stuff in. Email::Folder guesses based on the filename, and I don't know what it will guess here. Probably stupidly. Is it mbox format? Are you *sure*? If so, use Email::Folder::Mbox directly. -- rjbs
Re: I do not understand the error CGI that I get CGI using Email:: Simple
* fakessh [2010-05-04T08:20:14] > [Tue May 04 14:15:11 2010] [error] [client 90.30.250.62] Can't use string > ("fake...@fakessh.eu") as a HASH ref while "strict refs" in use at > /usr/lib/perl5/vendor_perl/5.8.8/Email/Simple.pm line 100., referer: This is a warning, not an error. > [Tue May 04 14:15:11 2010] [error] [client 90.30.250.62] Premature end of > script headers: webmail.pl, referer: It looks like your program just isn't behaving correctly. > >> > package PerlWebmail::Message; > >> > use base qw(MIME::Lite); > >> > use base qw(Email::Simple); > > > > Extending both those classes at once seems doomed to fail a lot. > > I can well use Email:: Simple in a CGI That is not the point. You are trying to create a class that inherits from two distinct and contradictory classes. This will fail. > >> > sub mail > >> > { > >> > my ($self, $message) = @_; > >> > my $obj = Email::Simple->new($message); > >> > print $obj->as_string; > >> > $Self->new; > >> > } > > > > You said $Self instead of $self. > > typographical error > the correcting operation in the class is ok Don't retype fake code. Show things that actually fail. Your program, as shown, makes no sense and will never do anything useful. -- rjbs
Re: I do not understand the error CGI that I get CGI using Email:: Simple
* fakessh [2010-05-03T20:55:20] > click error > > > when I am not MIME:: Lite > I find myself with a CGI error in httpd and browser. > I do not understand why What's the error? > > package PerlWebmail::Message; > > use base qw(MIME::Lite); > > use base qw(Email::Simple); Extending both those classes at once seems doomed to fail a lot. > > sub mail > > { > > my ($self, $message) = @_; > > my $obj = Email::Simple->new($message); > > print $obj->as_string; > > $Self->new; > > } You said $Self instead of $self. -- rjbs
ditching the wiki
The wiki isn't used. Unless someone objects strongly, I am going to get rid of it and replace it with a few static pages. -- rjbs
Re: spam category for Mail::DeliveryStatus::BounceParser
* Michael Stevens [2009-11-13T10:20:58] > I'm pondering adding a "spam" category to std_reason in > Mail::DeliveryStatus::BounceParser. > > This would record when a message was bounced because the recipient > considered it spam. > > Does this sound reasonable? Yes. -- rjbs
Email::Sender SMTP message
My master branch of Email-Sender currently adds a HasMessage role which gets applied to Success classes by the SMTP transports so that success can include the SMTP response after the "." This will allow a Postfix transport where success will have a ->queueid accessor. Anybody have thoughts on the idea or implementation? -- rjbs
Re: query regarding undocumented feature of Email::Simple
* Saurabh Hirani [2009-06-30T02:07:41] > So I looked at the source and deduced that if I have > $email->header_set($headername) i.e I don't give the value to set, as per > the internal working of the code, it actually deletes the header - which is > what I want. Is this an intended feature? Because it has not been > documented. Or is it a side effect of the current code, which might be > changed later? That's intentional. I use it all the time. I will document it. > It would really help the users of the Email::* project to know that as many > wouldn't look at the source. Thanks for developing such a wonderful set of > modules. Thanks for the email; it's always nice to know someone's getting some use from our code! -- rjbs
Re: Sending simple email
* "Roderick A. Anderson" [2009-06-06T17:50:13] > Ricardo SIGNES wrote: >> * Bill Moseley [2009-06-06T11:34:29] >>> I see the recommendation to use Email::Sender, but are there docs >>> available? >> >> You have sent this message at an excellent time! Email::Sender::Simple is >> potentially done, and I've written a quickstart guide, just last night. > > OK I'm hooked. How much more potential is there? :-) I think I've nearly got all our internal features into Simple. One that didn't make it is the archiver, which I will release later as a subclass of Simple. It basically says "after send_email, send AGAIN to ANOTHER transport, but throw away any failure." We use this to keep a copy of all the automated messages we send out without having to write code to do so explicitly. > I've been trying for a good and "simple" way to send email from Windows > and Linux systems. So E:S:S looks like it would fill the bill. In fact > I think I've tried and used (with differing success) Email::Sender and > friends but never came up with a consistent tool/method that worked for > both OS'. The current dev release of Simple doesn't yet try to use Sendmail transport, it just starts with SMTP. The next release will try Sendmail first, and fall back to SMTP. Real life experience with using it welcome! -- rjbs
Re: Sending simple email
* Bill Moseley [2009-06-07T19:07:34] > On Sat, Jun 06, 2009 at 12:01:40PM -0400, Ricardo SIGNES wrote: > > You have sent this message at an excellent time! Email::Sender::Simple is > > potentially done, and I've written a quickstart guide, just last night. > > > > http://rjbs.manxome.org/tmp/QuickStart.html > > Looks pretty good. Thanks. > > Any plans to add "simple" attachments? Like I mentioned, one of the > more common tasks for apps I work on seem to be attaching pdfs and > spreadsheets. Well, Email::Sender doesn't deal with attachments at all. It doesn't know what's in your message, beyond *maybe* looking at the to/from headers. Attachment stuff is all handled by Email::MIME. Can you give me an idea what the Perl you want to write would look like? > > You might like the BBC's Email::MIME::CreateHTML: > > > > Email-MIME-CreateHTML > > > > I think it does everything you want. > > I looked at that and I don't think it will create the plain text > version automatically from the html, but I'll take a look again. You're right, oops. In these circumstances I do one of three things: a) shell out to w3m (ugh!) b) only write the plaintext part, in Markdown, and use that to generate both (see http://search.cpan.org/dist/Email-MIME-Kit-Assembler-Markdown) c) use HTML::FormatText::WithLinks -- rjbs
Re: Sending simple email
* Bill Moseley [2009-06-06T11:34:29] > I see the recommendation to use Email::Sender, but are there docs > available? You have sent this message at an excellent time! Email::Sender::Simple is potentially done, and I've written a quickstart guide, just last night. http://rjbs.manxome.org/tmp/QuickStart.html > I tend to either send plain text or html formatted messages. If I > send an html messages and I don't also have a plain text version I > want to extract text and links from the html to make a plain text > part. You might like the BBC's Email::MIME::CreateHTML: Email-MIME-CreateHTML I think it does everything you want. -- rjbs
Re: Email::MIME issues
* Lyle [2009-02-16T10:27:01] > I guess the question really is, should Email::MIME take the single parts > headers, when only one part is added to the containing Email::MIME > object? Or should the single parts headers be lost, and the containing > Email::MIME objects headers be used. This is a bug in Stuff. The logic should be something like: if html xor text, start with that kind of part, with its attributes if html and text, start with multipart/alternative; attrs stay on parts if there are attachments, make multipart/mixed with parts [bodypart, att] Instead, the logic is much cruder, and is likely to produce incorrectly structured messages and, as you've found, trample attributes. My suggestion, if you don't want to really fix the code, is to not use Stuff. -- rjbs
Email::MIME::Kit released
After a long time and a bunch of rewriting, Email::MIME::Kit has been released. Here's what I posted in my journal about it: Building email messages is a pain. Even if you use a library to build the message string for you, you have to know a lot of crap and pay attention to a lot of details. If you know what those details are, it's a pain. If you don't know what they are, you don't feel any pain until later, when you find out all the ways things went wrong. It's more of a pain, too, when trying to set up multipart emails that are sent all the time, like subscription notifications, welcome messages, reports, and so on. I've seen a lot of awful solutions to this problem. My favorite awful solution was when I saw some code that took a complete, encoded MIME message and used it as a template. You'd have to pass it variables properly encoded with the right Content-Transfer-Encoding for the part into which the variable was used, sometimes passing two versions of one variable. Also, as I recall it used Python's format operator, so it all looked like a massive printf string. Then there are all the awful messages I get from vendors who build terrible MIME messages with broken encodings or text parts poorly converted from HTML (or empty!). Also, American Express has ignored my repeated complaints that while they provide a perfectly legible plaintext part, they do not render it, so I see all their template variables instead of my information. We set out to solve this problem internally a while ago, and I think it was a pretty big success. It made it very easy to throw together email templates that were maintainable, comprehensible, and that our web guy could edit easily. There were some problems, though. The biggest one, for me, was our code's reliance on one of YAML's more powerful features: tagging. We used tags to describe parts of the message in its YAML-based definition file, but dealing with YAML tags in Perl is still awful. Rather than pin our hopes to that improving, we've replaced that design. Actually, we've rewritten the entire library. I think it should be easier to use, easier to extend, and easier to understand. I'm very, very excited to start using it for all our internal messages. (Once again, this is Pobox saying, "We'd love to release this code, but not until we can rewrite it entirely based on the lessons we've learned. Normally I don't like that kind of thinking, but I thinks it keeps serving us (and the CPAN) well!) The library is called Email::MIME::Kit. It can be fairly significantly customized, but a fairly simple configuration, close to the stock one, works like this: You create a directory, which we call a message kit, or mkit, and put a bunch of files in it. These files are used in assembling the kit, and the most important is the manifest. We decide to write our manifest in YAML: --- validator: Rx renderer: TT header: - From:'"Customer Support" ' - To: '[% account.email_address.for_header %]' - Subject: 'Your Invoice, Number [% invoice.number %]' alternatives: - type: text/plain path: body.txt - type: text/html path: body.html attachments: - assembler: InvoicePDF attributes: { filename: invoice.pdf } When we send a customer a bill, now we can say: my $kit = Email::MIME::Kit->new({ source => 'share/msgs/invoice.mkit/' }); my $email = $kit->assemble({ account => $account, invoice => $invoice }); $transport->send($email, { ... }); The kit will validate the stash. Our example uses an [Rx](http://rjbs.manxome.org/rx/) schema, but writing a validator plugin is trivial. With that done, the message is assembled. The top-level part is assembled by the standard assembler, but the one attachment consults a custom assembler that retrieves or generates a PDF from our billing system. The only thing the designer needs to do is edit the text and HTML files in the message kit. These don't need to know anything about the fact that they're going into email. They will be properly encoded as needed. Even the headers are checked for non-ascii text and encoded as MIME headers (encoded-words) if needed. While the templates don't need to know they're going to be part of email, it can be useful. This is part of a manifest in Email::MIME::Kit's test suite: - container_type: multipart/related type: text/html path: better-alternative.html attributes: charset: utf-8 attachments: - type: image/jpeg path: logo.jpg The HTML template can get the Content-ID of any attachment easily enough, meaning that you can attach the images you want to reference in your HTML, rather than rely on web connectivity. Your template might contain: It's worth noting that this is all possible with the standard assembler, but wordy. I think we'll probably end up writing an assembler that optimizes for multipart/related HTML-and-content parts. Among the pieces that can be replaced are the KitReader (so yo
Re: Net-Server-Mail repo
* xavier.guim...@laposte.net [2009-01-15T12:31:40] > no problem for me, you can use GIT and delete svn files. Excellent. Earlier today I wrote a backpan-to-git importer. I imported all previous releases of Net-Server-Mail to git, then the unreleased changes, then a few minor fixes: http://github.com/rjbs/net-server-mail/tree/master I will probably be ready for a release in the next few days. Would you like me to ask you to cut releases as they're ready, or will you add me as co-maint on PAUSE? -- rjbs
Re: Net-Server-Mail repo
* Xavier [2009-01-15T01:26:31] > I have no other repository. I made a few changes on Net::Server::Mail > since it work for me (more than 1 million message a day) and I have no > demand to modify something. > > I've taken Net::Server::Mail in maintenance few years ago because there > was a lot of bug not corrected. Now, if you want to contribute, no > problem for me. I would be happy not to stay alone on this project. > I know that there is a problem with LMTP using Unix sockets (CPAN/RT) > but I can't reproduce it. Ok. I would like to delete the current Subversion installation on the box running emailproject.perl.org, and it sounds like you are not likely to set up a repo for this. I will import the existing history to GitHub. If you'd rather I don't, just say so and I will either not, or I will delete what I've already done. Once I've done that, I may check in some changes. If nothing else, I found some documentation problems yesterday. -- rjbs
Net-Server-Mail repo
Xavier, I feel like I already talked to you about this, but I can't find any record of it, so I'm guessing that I'm wrong. emailproject.perl.org/svn has been moved (mostly) to github. One of the few things I have not dealt with is Net-Server-Mail, because I am not one of its maintainers. I see that you made a commit to Subversion that never made it onto the cpan (0.06 of XFORWARD). Are you still working on this project? I'm actually looking at using it for some LMTP test work. If you have it in a new repo, great. If not, please let me know if you want me to export its history for you. I would like to turn off the svn server bits soon. Finally, if you're not using it any more, I would be happy to take over maintenance, release the changes, and fix any bugs I find while playing with it. Thanks! -- rjbs
wiki spam
As I suspected, the increase in wiki spam was because when I moved emailproject.perl.org from pep.pobox.com to mime.pobox.com, upgrading MediaWiki, I did not fix the configuration to require email verification. This has been fixed. Thanks to those of you who helped roll back vandalism. -- rjbs
Email::Sender 0.000 released
I still consider it fair game for big changes. I will be breaking it into a few dists soon. http://rjbs.manxome.org/rubric/entry/1706 -- rjbs
Email::Sender API approaches stability
I don't doubt that I'll hit a few glitches on the way to finality, but Email::Sender's API is getting pretty close to usable. Check it out and have a go: http://github.com/rjbs/email-sender Right now, normal usage is something like this: my $sender = Email::Sender::Transport::Sendmail->new; my $result = eval { $sender->send( $email, { to => '[EMAIL PROTECTED]', from => '[EMAIL PROTECTED]', }, ); }; my $error = @_; $result is either false or an Email::Sender::Success. If it's false, $error had better be an Email::Sender::Failure. Every failure has a 'message' and an array of 'recipients' and may have a 'code'. Sometimes the failure is a ::Multi, meaning that while sending was aborted, results were mixed. It contains other failures. They might all be of one subclass of Failure (Permanent or Temporary, for example) or there might only be one failure. Multi will have one contained failure when, for example: * four out of five recipients were accepted * ...but the SMTP transport is set to require 100% acceptance (default) This means that it is generally safe to say: $sender->send($email, $env); ...and let things fail if the email was not sent. This won't always be true, though, when there exist transports that are not capable of atomic transaction, like LMTP. They will be respond to something useful with ->does, probably. Anyway, if you really know what you're doing, you can also: my $sender = E::S::T::SMTP->new({ host => 'mx.your-isp', allow_partial_success => 1, }); my $succ = $sender->send($email, $env); In the event that 0>failure->recipients) { unsubscribe($addr); } I plan to write Email::Sender::Simple at some point to provide an interface a bit more like Email::Send's, complete with envelope defaults and true/false values with dying reserved for void context. Or something. I haven't given that much thought yet. Please let me know if you have feedback, as I'm definitely making progres toward calling this ready for upload (once there are docs (and more tests)). -- rjbs
Email::Sender: also not dead yet
The first thing that got me involved in PEP was Email::Send, which was largely unusable for serious email-related work becase it offered no mechanism for specifying an envelope sender or recipient. It had a bunch of other problems, some of which, we felt made fixing Email::Send in a backwards-compatible way a non-starter. Internally, at Pobox, we produced something called Email::Send::Mailer, which was a base class for Email::Send mail-sending plugins. It helped solve the problems, but required non-backwards-compatible changes to Email::Send and was, generally, a pretty grody workaround. Immediatley, we started talking about turning that work into a better Email::Send replacement, which we called (in the office and on the mailing list) Email::Sender. After a lot of delays, I've gotten Email::Sender into a state where it's nearly ready for release. I need to do a few significant things, but it's mostly ready to go. You can see it here: http://github.com/rjbs/email-sender Right now, it uses Mouse, unless Moose is already loaded. Mouse has zero prerequisites, so this is no more of an imposition than, say, Class::Accessor. Still, if it's going to ruffle too many feathers, I will release this code as Mail::Xmitter or something. Things that remain to do: * write up SMTP tests that can be used against different servers * prune no-longer-needed prereqs (see Makefile.PL for notes) * move SMTP_X, the persistent SMTP mailer, to xlib until it works * test mbox delivery * improve reporting of individual failures in sending, test it I welcome thoughts, criticism, suggestions, or bug reports. -- rjbs
Re: Email::Classifier: not dead yet
* Simon Wistow <[EMAIL PROTECTED]> [2008-12-01T20:45:53] > On Mon, Dec 01, 2008 at 07:15:20PM -0500, Ricardo SIGNES said: > > I may sit on it for another week or so, but unless I get some feedback on > > the order of "no, stop, this is crazy!" I am going to release it and start > > using it for things. Your input is hereby requested! > > This is more a point of philosophy and should probably be rephrased as a > question but - the original goal of the Email project when I helped > start it was to be as fast and lightweight as possible with as few > non-core dependencies as could possibly be gotten away with[*]. I have long been toying with putting much of this stuff under something other than Email, but it's sort of a moot point. There is plenty of garbage under Email::, of varying sorts. There's old-school "let's be simple" Email:: code that is so badly designed as to be unsalvageable and there's code from people who know nothing of the origins of Email:: and just thought it was the right place to put their big, reasonable email-related code. I think that having a top-level namespace that was supposed to all be for one related project's set of code was a failure when (a) a common noun was chosen (b) the actual project code was not built to clearly all go together. That said, I do keep that original goal in mind, and the email-specific Classifier code may well be Mail::Classifier or something. Until today, Email::Classifier's code was not Moosified. Making it Moose saved me a lot of crap code. Making it generic is also going to be useful, I know already. > As far as I can tell from a quick glance Classifier could be trivially > rewritten to use absolutely no non-core dependencies - I know Moose is > the flavour du jour but I'm still sad to the original goal abandoned. It's true, it could be, but extending it would be more of a pain, I'd need to test much more of its behavior, etc etc etc. I resisted Moose at work for quite a long time, maybe a year or more, before giving in and realizing how incredibly useful it is and how much time it saves me. Then there's Mouse, which gets you quite a lot of that, with no prereqs. Anyway, I don't mean to sound touchy, but I promise you that I resisted using Moose for anything in E/mail for a -long- time. I still plan to avoid it for the Email::Send replacement. My current guiding principle is "If you want to use it in some random code that deals with email, be as slim as possible. If you are going to use it for something that's really just about doing email and doing it well, don't mess around." If that means I'm supposed to use Mail::, then... so be it. It just seems like at some point Mail will start relying on Email and nobody will get it but the cabal. > But hey, this is your baby now (and has been for a long time) so I > suppose I have no right to grouse. Your grousing is always welcome! -- rjbs
Email::Classifier: not dead yet
I need to stop sitting on good ideas! Way back in February, someone posted saying, "how about that email classifier we want to replace the awful awful BounceParser?" I replied with some of my own ideas, including an implementation. Then I did nothing. http://www.nntp.perl.org/group/perl.pep/2008/02/msg288.html I finally need Email::Classifier for something else, and I brushed off the code. I realized that as written, only the tests were email-specific. I made it a generic thing classifier, improved the interface just a tiny bit, and made it use Moose. You can find Classifier here: http://github.com/rjbs/classifier/tree/master I may sit on it for another week or so, but unless I get some feedback on the order of "no, stop, this is crazy!" I am going to release it and start using it for things. Your input is hereby requested! -- rjbs
Re: porting to github
* Ricardo SIGNES <[EMAIL PROTECTED]> [2008-11-13T21:42:39] > I will begin moving PEP SVN to github shortly. I will not move the few things > that I did not put there myself, and will contact those authors if I can. Email-Applications Email-Base Email-Corpus Email-Filter-SpamAssassin Email-Sender Email-Stuff Net-Server-Mail bundles All but the above have been converted. Email-{Base,Sender} are likely to be scrapped and started over. Email-Corpus, I need to review. Email-Applications is 'buscador' which is abandonware of a higher order than Email-Store, and will be abandoned. Email-Stuff is slightly out of date, and will be updated and converted. 'bundles' will be converted into something more useful, like a bundle builder with some config. The remaining code, EFSA and NSM, was not under my care to begin with, and I will ask the authors their wishes... actually, that's what this email is! Uwe, Xavier, shall I port that code to github with the rest of the PEP repo, or would you like to take care of hosting the repo yourself? -- rjbs
porting to github
I will begin moving PEP SVN to github shortly. I will not move the few things that I did not put there myself, and will contact those authors if I can. -- rjbs
Re: Strange Behavior in Email::Valid for email addresses w/newlines
* Justin Simoni <[EMAIL PROTECTED]> [2008-10-08T05:06:11] > I'm having an interesting case, where email addresses with newlines > are seen as valid, but carriage returns and/or carriage returns, with > a newline aren't seen as valid: Interesting... > That regex in Email::Valid is scary, That's the understatement of the month. > although this line seems a little suspicious: > > $RFC822PAT =~ s/\n//g; That *is* a bit odd, but my first instinct is to think that it does't matter. I'll bring your tests in and have a look. I imagine it will be something stupid. > set to, "false" by default - so I'm assuming that no attempts at > making the address is corrected are done. The URL to the Book's code > examples is 404'ing and it's time for me to go to sleep :) That URL should be removed. -- rjbs
Re: new server: some organizational questions:
* Ricardo SIGNES <[EMAIL PROTECTED]> [2008-06-24T21:23:07] > For boring reasons, I'm moving emailproject.perl.org to a new box. This is basically done. By morning, DNS propagation should be done. In the meantime, uh... refrain from comitting unless you know you're hitting the box known as mime.pobox.com! -- rjbs
Re: bug in MIME::Entity make_singlepart
* Dave O'Neill <[EMAIL PROTECTED]> [2008-06-27T10:30:10] > On Thu, Jun 26, 2008 at 01:37:09PM -0400, Ricardo SIGNES wrote: > > Content-Type: multipart/related; boundary=xyzzy; type=foo > > Content-Type: text/plain; boundary=xyzzy; type=foo > > The docs for make_singlepart say "Also crunches 0-part multiparts into > singleparts", so the first option is probably the safest for backwards > compatibility. However, it does seem nonsensical to convert an empty > multipart/* part into a text/plain. Is this a case that's frequently > encountered? If not, maybe it's OK to change that behavior. I think that it's documented, so it should keep working. If nobody has asked for it to throw an error, it either isn't coming up or it's coming up and people accept it. I'd suggest adding an error-generating branch (with a different method name or an arg) or just removing the entire Content-* header set (since there is no content!) but I wouldn't need to use it, and who knows if anyone else would. I think the only problematic behavior is the bizarre (and presumably illegal) leftover parameters. -- rjbs
Re: I hate Unicode
* Karen Cravens <[EMAIL PROTECTED]> [2008-06-26T15:38:17] > (Dangitall, if 7-bit ASCII was good enough for Fidonet, it ought to be good > enough for email. Kids these days.) > > I'm getting this when I try to retrieve > $SomeEmailMimeMessage->header('subject'): > > Unknown encoding "X-UNKNOWN" at /usr/lib/perl/5.8/Encode.pm line 166 Wow. I had never noticed this bit of HORRIBLENESS before. > Well, crud. But here's the funky thing, that I'm not quite getting: it's > not happening when I *create* $SomeEmailMimeMessage. I could deal with > that. It's happening when I *look* at that header. Why is that, and how can > I trap it (or ideally, detect it when I first load up the message)? Email::MIME, once again, is shown to be useful for a very, very small set of email. That is, email that is not wrong; all correct email won't work either, but this email is failing because it's not to-spec. Probably. I'm not sure if the encoding in a encoded-word needs to be in a registry somewhere, and whether X-UNKNOWN is. Anyway, if you must solve this while still using Email::MIME, try the following awful incantation, which happens to probably work: my @values = $email_mime->Email::Simple::header('foo'); > Or should I not be handing Email::MIME something that's gone through > encode('unicode', $mailstring) at all? It's mail that was improperly stored > in an SQL-ASCII database, and is going into a UNICODE database, so it's a > one-time conversion utility on what may well be lightly-fried data. If this > is something that's never going to come up on fresh mail, I can code for > the finite number of exceptions in this database. Headers *must* be encoded into a seven bit format. I have no idea what "unicode" means as the first arg to encode, but I doubt that it's 7-bit safe. You'll want to use Encode::MIME::Header, which means you'll need to have a utf-8 string first. my $octets = $octets_that_you_know_are_latin_3; my $string = Encode::decode('latin-3', $octets); my $header = encode('MIME-Q', $string); my $email = Email::MIME->create( header => [ From => $header ], ... ); You need to have a utf8 decoded character string before you make the header value. -- rjbs
Re: bug in MIME::Entity make_singlepart
* Ricardo SIGNES <[EMAIL PROTECTED]> [2008-06-26T13:37:09] > Demonstration attached. Thanks, ezmlm. Here it is: use strict; use MIME::Parser; # my $entity = MIME::Parser->new->parse_open('bizarre.msg'); my $entity = MIME::Parser->new->parse(\*DATA); sub cleanup_mime { my ($entity) = @_; foreach my $part ($entity->parts) { cleanup_mime($part); $entity->parts([]); } } cleanup_mime($entity); $entity->make_singlepart; print $entity->as_string; __DATA__ Received: from indigo.pobox.com (indigo.pobox.com [207.106.133.17]) by chiclet.listbox.com (Postfix) with ESMTP id 2AE91214A41 for <[EMAIL PROTECTED]>; Tue, 24 Jun 2008 01:22:44 -0400 (EDT) Received: from vip-2fed93075f2 (unknown [116.60.133.101]) by indigo.pobox.com (Postfix) with SMTP id 4DE1A6BF4D; Tue, 24 Jun 2008 01:22:30 -0400 (EDT) From: "ÕÅÏÈÉú"<ÕÅÏÈÉú> Reply-To: "[EMAIL PROTECTED]"<[EMAIL PROTECTED]> To: "[EMAIL PROTECTED]"<[EMAIL PROTECTED]> Subject: =?gb2312?B?uePW3crQwaq3or34s/a/2sOz0tfT0M/euavLvg==?= Date: Tue, 24 Jun 08 13:22:18 +0800 MIME-Version: 1.0 Content-type: multipart/related; type="multipart/alternative"; boundary="=_NextPart_000_0015_1963AAAC.4C2B0004" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 Message-Id: <[EMAIL PROTECTED]> This is a multi-part message in MIME format. --=_NextPart_000_0015_1963AAAC.4C2B0004 Content-Type: multipart/alternative; boundary="=_NextPart_001_0016_1963AAAC.4C2B0004" --=_NextPart_001_0016_1963AAAC.4C2B0004 Content-Type: text/html; charset=gb2312 Content-Transfer-Encoding: base64 PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMCBUcmFuc2l0aW9uYWwvL0VO Ij4NCjxIVE1MPjxIRUFEPg0KPE1FVEEgY29udGVudD1odHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29t L2ludGVsbGlzZW5zZS9pZTUgDQpuYW1lPXZzX3RhcmdldFNjaGVtYT4NCjxNRVRBIGh0dHAtZXF1aXY9 Q29udGVudC1UeXBlIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD1nYjIzMTIiPg0KPE1FVEEgY29u dGVudD0iTVNIVE1MIDYuMDAuMjkwMC4zMzU0IiBuYW1lPUdFTkVSQVRPUj48L0hFQUQ+DQo8Qk9EWSBz dHlsZT0iRk9OVC1TSVpFOiA5cHQ7IEZPTlQtRkFNSUxZOiDLzszlIj4NCjxQPsT6usMhPEJSPiZuYnNw OyZuYnNwOyCxvrmry77XqMPFzqq498Oz0tfJzLvyyfqy+rOnvNK0+sDtu/XO7734s/a/2rGoudihoiAN CsnMvOyhorWl1qShosjrstahorGoudjK1tD4oaLNz7O1oaLW0LjbLLT6sOy499bWsvq12NakyukuwarP tcjLo7rVxc/Iyfogyta7+jEzNjMyMjc4MzMyJm5ic3A7IFRFTDAyMC0zNzIzMjYwNiANCjYxMDMwOTY0 IEZBWDAyMC02MTAzMDUxNSC12Na3o7q549bdytDM7LrTx/jR4MHrwrcxMjC6xb3w0eC088/DIMjn09C0 8sjFx+u8+8HCITxCUj48L1A+PC9CT0RZPjwvSFRNTD4NCg== --=_NextPart_001_0016_1963AAAC.4C2B0004-- --=_NextPart_000_0015_1963AAAC.4C2B0004--
bug in MIME::Entity make_singlepart
I thought I'd cc the list since this is sort of a weird, fun bug. Sometimes, when collapsing a message into single part, the C-T is horked up. It starts as: Content-Type: multipart/related; boundary=xyzzy; type=foo ...and ends as: Content-Type: text/plain; boundary=xyzzy; type=foo Erk! Boundary? Type? What? Demonstration attached. -- rjbs
Re: parse out the email body
* Basil Daoust <[EMAIL PROTECTED]> [2008-06-25T22:23:07] > > You want something like: > > > > my $root = Email::MIME->new($message_text); > > my ($html_part) = grep { $_->content_type =~ m{^text/html} } > > $root->subparts; > > > > my $encoded_html = $html_part->body; > > my $html = Encode::decode($charset_from_content_type, $encoded_html); > > > > Thanks, that works though the way you wrote that I thought > "$charset_from_content_type" was a MIME set value when new was run. Well it > doesn't appear to be, but no problem. No, you have to do something like: use Email::MIME::ContentType; my $ct = parse_content_type($part->content_type); my $charset = $ct->{attributes}{charset}; > Now this works and so does pulling "Text/plain". Text plain is what I want, > so when its available great. I'm guessing that to cleanup text/html, I > should use something like HTML::Parser? I don't know what you mean by "cleanup." -- rjbs
Re: parse out the email body
How many times did you send this message to the list? * "Basil A. Daoust" <[EMAIL PROTECTED]> [2008-06-24T14:59:34] > I want to be able to read a MIME encoded email file (Thunderbird) and have > a way of returning just the VIEWABLE body part. "Viewable" isn't defined by any RFC. What do you mean? > I tried Email::Simple but I get all the MIME encoding and both the TEXT and > the HTML parts. That won't work, because Email::Simple doesn't understand MIME. > I tried Mail::Internet but then besides everything that Email::Simple gives > I also get all the header data as part of the body. That won't work, because Mail::Internet doesn't understand MIME. > my $obj = Mail::Internet->new([ /\n/, $message ] );my $body = > $obj->body; > print @$body,"\n"; > That returned the entire email, headers and everything, so much for > getting the body. I think you meant to put a "split" after the [. That's why it put everything in the body. You want something like: my $root = Email::MIME->new($message_text); my ($html_part) = grep { $_->content_type =~ m{^text/html} } $root->subparts; my $encoded_html = $html_part->body; my $html = Encode::decode($charset_from_content_type, $encoded_html); -- rjbs
new server: some organizational questions:
For boring reasons, I'm moving emailproject.perl.org to a new box. My plan is to just port over the existing Subversion repository and MediaWiki. If you want to demand major changes, this is the time to do it. -- rjbs
Re: Problem with attachment, 1 becomes 10
* Mikael Bonnier <[EMAIL PROTECTED]> [2008-05-04T20:49:51] > I'm trying to write a program that can send out mails to companies where > I'm interested in working. These mails are the same except that the company > name is switched. My CV should be attached as a pdf-file. Let me rephrase your question: I am trying to spam recuiters. Can you help? No. -- rjbs
Re: Announcement Test::SMTP module released
* Jose Luis Martinez <[EMAIL PROTECTED]> [2008-04-28T14:24:40] > I released a week ago Test::SMTP. This module is a helper for writing > tests for SMTP servers. I basically announced it on the qpsmtpd list, but > I've just found this list, so I'm reannouncing: Thanks for the note. I saw it in the CPAN uploads, and have wanted to check it out, but so far I haven't had the time. -- rjbs
Re: validating email addresses
* Dave Howorth <[EMAIL PROTECTED]> [2008-04-17T05:42:59] > I use Email::Valid to check the syntax and MX record. I'm wondering if > there is any best practice for the next steps: > > - verify the existence of the mailbox Don't. Email verification probing is basically abuse, in my opinion. The correct thing to verify is that a person can read the mail sent to the address. In other words: > - send an email requiring a reply The common practice these days is to send an email that has a link in it. Sending one that also accepts a reply is just bonus. I have some code that helps a little with this, but there's not much to it. $user->set_verification_token($random_value); $link = generate_link_with_token($random_value); $email = generate_email_with_link($link); send($email, to => $user); Then when you get a hit at the link with that random value in it, the user is verified. I'm sure this has been implemented a million times on CPAN. My implementation is in Rubric. -- rjbs
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-25T08:25:54] > I want to get it on demand. Not extracting all possible information just in > case it may bee needed. It could be hundreds of different things. > > Ok. So I could let $msg_id be a object that computes itself on > stringification. My first thought is YAGNI: You Aren't Gonna Need It. That is, I predict that there are not hundreds of things that any given classifier is likely to report on, only a few, which would not be too expensive to compute up front. Let's say I'm wrong: I could see this being nice and simple: $report->details->{helper}->orig_msg_id; Where the "helper" is an object that can compute a bunch of stuff as needed, rather than using a lot of thunks. I don't even mind if "details" is sometimes set to an object that can be accessed as a hash, so that you could say $report->details->orig_msg_id; In other words, the semantics of "details" can change a bit, but the semantics of "report" should not. Here is my concern in more detail: my $classifier = E::C->new({ ... classifiers => [ qw(-Bounce -SpamAssassin -Autoreply -ClamAV -Cloudmark) ], ... }); my $report = $classifier->classify($email); if ($report->spam_score > 5) { ... } Well, first, where does "spam_score" come from? I see a few options: 1. Each classifier can declare that its reports are of a specific class. So, E::C::SpamAssassin reports with Email::Classifier::Report::SpamAssassin. That has a "spam_score" method. The problem here is that if the result was a Bounce report, there will be no spam_score method and now you've thrown a method missing exception. Having to eval every method call on $report seems awful. 2. Each classifier can import more methods into the report class. So, when loaded, E::C::SA adds a spam_score method to Email::Classifier::Report. Now every report has a spam_score method. Unfortunately, -Cloudmark wants to also. There is a conflict, so we can't use the same name. Or maybe we can: the methods are created and they do nothing unless the report is of a class known to handle the method. This means that every added method needs to be able to return undef (or something else) if the report doesn't handle it. That's weird to begin with. Another problem is that SpamAssassin and Cloudmark may both return spam_score, but with very different meanings. A "10" on SpamAssassin is quite high, but quite low for Cloudmark. So, maybe we force all plugins to use unique names. spamassassin_spam_score and cloudmark_spam_score. We still might see conflicts, especially if we have *two* SpamAssassin classifiers loaded with different config: # This uses the second term as a classifier name, as dave0 originally # suggested. classifiers => [ [ -SpamAssassin => lax=> { hits_required => 10 } ], [ -SpamAssassin => strict => { hits_required => 5 } ], ], Well, now we could prefix all added methods with the moniker: $report->lax_spamassassin_spam_score 3. Like #2, but optional and isolated: In other words: my $classifier = Email::Classifier->new({ ... classifiers => [ ... ] extended_reports => 1, }); If extended_reports is set, a new class is created, subclassing Email::Classifier::Report, with methods mixed in. Other Email::Classifiers in the same process, then, are not affected. I think this is a necessary enhancement to option #2. None of these options, though address the problem of aggregation: the analyze method, which does not skip rejects or short-circuit at the first match, returns a ResultSet object which nicely aggregates the result data. It doesn't have a type or details method, so I guess it may not be a big deal if it doesn't get any of the other extra methods. Option #3 strikes me as sort of overly magical, but I can see its appeal. Anyone else have thoughts? -- rjbs
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-25T05:04:56] > I still want to have methods dependent on the type of classifier. The > orig_message_id may have to scan the message, searching for the id. But > it should not have to do so if the message_id isn't going to be used. > > Other examples are orig_message_head that should try to extract the > message_head from what ever format used. Maby giving access to classifier? > > my $report = $classifier->classify($email); > if( $report->type eq 'dsn' ) > { > print "DSN for ".$report->obj->orig_message_id; > } Under the code I sent, this could be implemented as: my $report = $classifier->classify($email); if( $report->type eq 'dsn' ) { print "DSN for " . $report->details->{orig_message_id}; } There's no need to keep the classifier around or have a reference to it. The classifier would just end up doing: if ($is_bounce) { return $self->match({ orig_message_id => $msg_id, type=> $computed_type, ... }); } -- rjbs
Re: Email::Classifier
* Ricardo SIGNES <[EMAIL PROTECTED]> [2008-02-21T23:38:18] > I think my main concern relates to the way that one would use Classifier, and > what it would return. I imagine this as a very basic use case: > > my $classifier = Email::Classifier->new({ > classifiers => [ ... ], > ... > }; Rather than just wave my hands and say, "Uh, something like this," I have put together a quick implementation of Email::Classifier as I described it. Something not implemented, from a post in July by Dave O: > my $r = $parser->parse( $email_abstract ); > if( $r->is_gpg_signed() ) { # let plugins provide new methods to > # result object (inheritance? Sub::Exporter?) > if( $r->valid_signature_by('0x12345678') ) { > # Yes, we fully trust the message now > eval $email_abstract->get_body(); ...the ability to add report methods. I think that needs more thought. Tarball attached. -- rjbs Email-Classifier-0.001.tar.gz Description: application/tar-gz
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-18T16:45:32] > http://jonas.liljegren.org/perl/dist/Email-Classifier-0.01.tar.gz I finally got around to flipping through this, tonight! I think my main concern relates to the way that one would use Classifier, and what it would return. I imagine this as a very basic use case: my $classifier = Email::Classifier->new({ classifiers => [ ... ], ... }; my $report = $classifier->classify($email); if ($report->type eq 'bounce') { my $details = $report->details; log("got bounce from $details->{remote_mta}); if ($details->{status_code} =~ /^5/) { ... } } A classifier returns a report, which has a globally fixed API. It can provide details, which are defined by the winning classifier. The report is generated by the specific classifiers set up in construction. The first one to issue a report wins. If no classifier matches, we return false. This will handle most use cases, I think. It will also be easy to say: my @reports = $classifier->all_classifications($email); ...to get every report, in the event that we think ones after the first hit will be useful. If a $report has one and only one type, we could say: my $report_set = $classifier->all_classifications($email); my @bounce_reports = $reports_set->of_type('bounce'); ...but now I'm getting into exotica. I think the really nice simple bit of design is that Email::Classifier runs an email through a set of specified classifiers, stops when it gets a hit, and returns an object with a globally constant API. It's easy, then, to say that the classifiers for Email::Classifier are, in turn, Email::Classifiers. So, Email::Classifier::Bounce is just an Email::Classifier built up of all its Email::Classifier-compliant plugins for specific bounce formats. This also avoids all the is_* methods, conflict detection, AUTOLOAD, and typeglob wrangling in the code you posted. I also think that totally ditching Mail::DeliveryStatus::BounceParser is a good start: it will be a nightmare to fuss with or extend, even in the form you've pruned it down to. So, if I am not full of crap, the two big tasks are: 1. implement Email::Classifier, which should be really easy 2. implement various useful classifiers, starting with some bounce processors A good precursor to (2) is to build up a good corpus of messages. Thoughts? -- rjbs
Re: Email::Classifier
* Jonas Liljegren <[EMAIL PROTECTED]> [2008-02-17T19:50:52] > The MD::BounceParser seemed to bee a bit disorganized and had seems to > have the assumption that you only would want to use it for finding out > email-addresses to remove from send-lists. That is definitely the reason it was made. It was also written ages ago, and it's difficult to extend to add things like, "reason = they think it's spam." > Depending of what you want to do, you will treat these cases differently. > A hard bounce is a response that the mailbox no longer exist. A soft > bounce is a mailbox over quota or some other thing that may not be a > permanent condition, but could be. A transient bounce is just a status > report and not a failure. But I still want to catch those transient > error messages. They should certainly not go into the request queue, but > they should still be parsed and used to update the status of the > original email and receiving user. This can be done then there is a > custom built web interface for the email handling. "Hard" and "soft" are definitely well-established terms for describing bounces. I think that "transient bounce" is a weird choice, though. It's not a bounce, and if it was, a transient bounce would sound, to me, like a soft bounce. It's just a DSN that isn't a bounce reporting DSN. I agree, of course, that understanding these is a good idea. > I have set up Email::Classifier. I just took all the code of > MD::BounceParser and converted it. It uses modules that loads on > demand. Most of the code ended up in the Email::Classifier::Bounce > module, that still needs a lot of cleaning up. I look forward to having a look at it! I will try to do so today or tomorrow, as time permits. Thanks very much! -- rjbs pgp5E1CeCNFZv.pgp Description: PGP signature
Re: IMAP and message modification
* "Roderick A. Anderson" <[EMAIL PROTECTED]> [2008-01-14T17:09:38] > I've read (most of) the IMAP RFCs and searched and looked at many of the > CPAN modules that deal with IMAP but I'm having no success finding what > I'm looking for. I'm probably using the wrong terminology. Have you tried using Mail::Box::IMAP4 to get the message, alter it, and then call ->update on the box? -- rjbs
Email::Corpus building
To build a corpus distribution, you'll use Module::Install::EmailCorpus. It will cause 'make' to build an SQLite database out of the contents of ./corpus (see below) and will cause that file ot be installed to the package's sharedir. ./corpus will contain one directory per message, each of which will contain a "message" file and a "meta" file. The "meta" file will contain a single YAML document with a sequence of data about the message. Every message will have a guid. (Module::Install::EmailCorpus will, I hope, provide a 'make guidify' to add guids where needed, and will refuse to 'make dist' if not every message has one.) A sample meta file might be: --- guid: 3F2504E0-4F89-11D3-9A0C-0305E82C3301 tags: - mime - crlf:lf - 8bit Email::Corpus::Magic: magic: 1 moon: full The meaning of package-named mapping entries are left up to the owner of the namespace to define, by convention. -- rjbs
Email::Corpus
Two emails. First, a sample of what I think it will look like: use Email::Corpus; use Test::More; use Hypthetical::DKIM::Tester qw(check_dkim); my $collection = Email::Corpus->search({ package => [ -Core, -DKIM ], name=> $name, tags=> [ qw(mime tnef) ], filter => sub { exists $_->meta->{dkim} }, }); plan tests => $collection->count; while (my $email = $collection->next) { my ($pkg, $guid) = map { $email->$_ } qw(package guid); is( check_dkim($email->abstract), $email->meta->{dkim}, "expected dkim result on $guid in $pkg", ); } -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Justin Simoni <[EMAIL PROTECTED]> [2007-12-19T20:14:58] > You know, I could really use some help with Dada Mail - the last time > this thread was active, my message I sent telling people about it was > never posted. > > Anyways, the next alpha release of Dada Mail is looking pretty killer. > If anyone is interested in helping with development, let me know what > your pet project may be. I had a glance at it, but I need to look more. Maybe I can help make it CPANable. I work at IC Group, which runs (among other services) Listbox.com, which is a really nice MLM. I'm looking to abstract and publish parts of that system as a generic MLM framework, of which Listbox would be one implementation. I'm hoping that I can release a lot of the cool stuff we have, without the impossible (and unappealing) task of trying to make Listbox itself a complete, released free software product. I will post about that work shortly. Maybe it will turn out that Dada and Listbox (and other things) are similar enough that building this framework really will be useful. > One of the projects I'd really love help on is getting a proper, make, > make test, make install distribution and having it hosted on CPAN. > Right now, I've just been working on the test suite, which has about > 5,000 tests already. I think that if you make it a CPAN distribution, you will get more people to help faster. Being able to say "install this dist" and have it Just Go is a huge time-saver for potential hackers, in my experience. I know that *I* certainly give up on a lot of Perl code when I see it isn't packaged standardwise. > but I've managed to do quite a heap of work already. A lot of the code > isn't up to the pep spec, but a lot of the code was written before > pep's recommended CPAN modules existed. That's OK, a lot of PEP's code isn't up to PEP standards. Actually, a lot of it wakes me up at night in a cold sweat. Improving it is half the fun. (The other half is ... well, I'll figure it out sometime.) > Another project may be to get it working with those modules - for example, > replacing the MIME-Tools stuff with other things. One of the main goals of > Dada Mail is to have it installed without needing any XS perl modules, so > people who do not have CPAN (or don't know how to use) or a compiler can > install it. MIME-tools is awful in some ways that I know about and some that I don't, but it's also got a LOT of edge on Email::MIME in many places. I'd give a look at Mail::Message and see if it looks worth upgrading. It very well may not. P.S., Hugo Ball rules. -- rjbs
(non-)progress on Email::Base and Email::Sender
I'm tired of trying to fix hard-to-fix issues in Email::Send, and tired of saying, "Yeah yeah, Email::Sender someday soon blah blah." So, I'm doing a bit more work on it. Here are my current thoughts: Email::Base is good enough for now. All it does is provide ->throw, which is enough. Someday, I imagine there will be more to it. The big thing it doesn't do that we talked about is a generic constructor and attributes. This isn't a big thing, but it's starting to mildly irritate me as I work on Email::Sender. I don't want to just "bless $arg => $class", and I don't want to say, "all guts are available to all subclasses" or "stick your subclass's guts in $self->{__PACKAGE__}. These all suck. Also, I've refactored "args for the Sender" from "envelope data." Given a mail with a "prec" send-time option, this is still correct: $sender->send($email, { to => $rcpt, from => $sender, prec => 'bulk' }); ...but it is turned into: $sender->send_email( $email, { to => $rcpt, from => $sender }, { prec => 'bulk' }, ); Now, you end up writing: sub send_email { my ($self, $email, $env, $arg) = @_; ... } Blech. What I really want is something we've long talked about internally at work, and something that DaveO and I talked about once or twice at YAPC: messages with attached envelopes. Email + Envelope = Delivery. The you say: $sender->send($email, { to => $rcpt, from => $sender, prec => 'bulk' }); ...and it becomes: $sender->send($delivery, { prec => 'bulk' }); If you already have a Delivery ready to go, you pass it to the $sender->send to begin with. The way I see it, a Delivery does the Email and Envelope roles. It has-a Email and has-a Envelope, and delegates the roles to them. This (plus the pair of fuzzy antler headgear I received for Christmas) is making me start to decide that I'm going to say "screw it" and use Moose. If Email::Sender is a Moose object, it becomes very simple to deal with the attribute issues. I also think (but cannot yet prove) that it will make Email::Sender::Wrapper trivial to do away with, letting Email::Sender::Failable operate directly on concrete Sender classes. I will start work on this in the next week, and it should be very easy. If you have objections, voice them now. My only concern at the moment is that I want the Email role to be easy to grow in the future. I'll spill my guts on some of that now: I want method naming to be uniform across all of these classes, and I think it may have to be set/get. Here is why: headers. It is very convenient to say: $email->header($header_name); This is also convenient: $email->header($header_name => $value); This starts to get weird: $email->header($header_name => $value, $value2); This sucks: $email->header($header_name => undef); I'm also not crazy about "well you use set to set it to anything except for nothing, which requires delete." I could be convinced, though. There's always "setting multiple values requires an array ref, and [] clears." I don't know, I welcome thoughts on the matter. There will be a EmailHeader role, I think. I'm not sure how it will work yet, either. I will probably steal or at least think about stealing from Mail::Box, which does some cool stuff with how bodies and headers interact. Ok, this message is now officially too long. I look forward to hearing your thoughts. -- rjbs
pep and 5.005
I was mildly surprised when I saw Michael Schwern's recent announcement that he would no longer be supporting 5.005. I was happy to see it, though, because my plan has been to drop 5.005 support, with perl 5.010 coming out. This will mean, at the very least, that 'warnings' and 'our' will no longer be taboo. I'm not going to go out of my way to break things, but as modules get worked on, I will no longer shy away from breaking compatibility with perls more than seven years old. Email::Date::Format, today's new module, requires 5.006. In a few days, MIME::Lite will require it (to drop Email::Date) and I'm sure more will follow. That's all -- I hope everyone's upgraded sometime in the last seven years. -- rjbs
new Email::Abstract!
At long last (and about 3 months past due, says my OmniFocus) I have released a new stable release of Email::Abstract. This release should be significantly saner, with better facilities for a plugin to declare when it will work and when it should be ignored. Email::Abstract is likely to be the email representation passed around in Email::Sender, which should be the next thing to get some attention. Please let me know if you have any problems or concerns about it. -- rjbs
the big todo list, posted
http://emailproject.perl.org/wiki/Big_Todo_List I've dumped my previous mail on this topic to the wiki, in response to the occasional, "What can I do?" mails and privmsgs. So far, these haven't led to much, but... well, a fellow can hope. -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Karen Cravens <[EMAIL PROTECTED]> [2007-09-19T11:53:13] > sinus yuck and can focus on finishing the install, we can move over there. > (Guess I could just turn on the daemon and hope for the best...) Of > course, once I get off the Sudafed+Dew I probably won't be so inclined > toward lengthy rants. No, I think enough of us have a vested interest in seeing this kind of thing done properly. > (I see that Boulevard isn't available in Chicago, so I reckon I'll be > bringing another trunkful to YAPC2008. Not sure about Flying Monkey, but > probably that too.) Seriously, I still think about that Boulevard beer from time to time. I wonder if my local beverage distributor could acquire a case for me. -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Simon Wistow <[EMAIL PROTECTED]> [2007-09-14T18:39:33] > As for Perl MLMs - have you looked at Siesta? It's a bit rough and ready > in some places but I've been running mailing lists off it for years. On the topic of Perl MLMs, I work at Listbox.com, something of an "enterprise" MLM. I'd like to help get what bits of its code can be made useful for reuse out into the world at some point, so anyone working on this stuff should feel encouraged to bug me (on the list) for thoughts, patches, or code. -- rjbs
Re: Email::Store is dead! Long live Email::Store!
* Christopher Nehren <[EMAIL PROTECTED]> [2007-09-14T17:00:51] > We've got a (mostly empty for the moment) repo at > http://www.coitusmentis.info/repos/email-archive/ , hosted on my home cable Would you like to host this on emailproject.perl.org, on a nice Sun server at a datacenter? -- rjbs
Re: Messages won't import into MySQL
* Chris Miller <[EMAIL PROTECTED]> [2007-08-03T19:26:08] > Most messages import just fine, but I'm running into more and more messages > that don't make it. Sendmail complains with the following : > > dsn=5.3.0, stat=unknown mailer error 255 > > The message content gets inserted into the 'mail' table, but the message ID > never makes it into the 'mail_date' table. Unfortunately I use the mail_date > table to sort messages by date. I can have a brief look, but I have never, ever used Email::Store. I only updated a few CLEARLY busted things to bring it under the care of PEP. I think there are some others here who have more experience with it. -- rjbs
MIME-Lite in pep svn
As those of you on pep-checkins saw (sorry for the huge noise), MIME-Lite is now in the PEP subversion repository. I've made a new dev release with zero code changes. I'm not a fan of MIME-Lite, as it's fairly buggy and not all that lite. I'm not really interested in making it the best MIME generator around (though feel free to do that), but I would like to reduce its bugginess, since a lot of things use it. As those of you on pep-bugs know (sorry for the brief outage; my bug aggregator script is lame), there are about 120 bugs against email-related modules. 31 of those (that is: 25%) are filed against MIME::Lite. Testing so that 3.020 can be released would be great. Commits to fix bugs would be great. Be great! -- rjbs
base exceptions
Right now, Email::Sender uses -Sender::Failure, and two subclasses of that. The only core exception (beyond the root class) that I can imagine is Unimplemented. Thoughts? -- rjbs
Re: Email::Base 0.000
* Hans Dieter Pearcey <[EMAIL PROTECTED]> [2007-07-24T22:00:15] > On Tue, Jul 24, 2007 at 07:46:36AM -0400, Dave O'Neill wrote: > > Using '::Bar::Baz' is probably a better choice. Most Perl people know > > what a :: prefix means for namespacing, but only Catalyst people will > > find '+Bar::Baz' intuitive. > > I think this is worse, honestly; I wrote a whole reply to you assuming that > you meant ::Bar::Baz would have Email::Exception prepended, because that is > how I instinctively parsed it. ->throw('Fully::Qualified::Name'); ->throw('-Under::EE'); (a) hypen means "tacked on to previous thing" like "hyper -active and -trophic" (b) it lets you skip quoting (c) which means less typing for the common case ->throw(-Under::EE => { }); I dunno, it's late and I'm about to sleep, but it's a thought. -- rjbs