Debugging strangeness in To: field

2011-04-10 Thread Matthieu Lemerre
On Wed, 6 Apr 2011 12:58:25 -0600, Mark Anderson  
wrote:
> Hello all,
> 
> Do you have any hints about how I could figure out why gmime doesn't
> like this To: list?
> 

Hi,

I have encountered this problem before; see id:"87ipzvk2xh.fsf at free.fr".

Basically you have to upgrade gmime, but the debian package is not
up-to-date.

Matthieu


Re: Debugging strangeness in To: field

2011-04-10 Thread Matthieu Lemerre
On Wed, 6 Apr 2011 12:58:25 -0600, Mark Anderson  wrote:
> Hello all,
> 
> Do you have any hints about how I could figure out why gmime doesn't
> like this To: list?
> 

Hi,

I have encountered this problem before; see id:"87ipzvk2xh@free.fr".

Basically you have to upgrade gmime, but the debian package is not
up-to-date.

Matthieu
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Debugging strangeness in To: field

2011-04-06 Thread Mark Anderson
On Wed, 06 Apr 2011 13:35:44 -0600, Mark Anderson  
wrote:
Hello All,

> It is rather painful that I can have a lot of recipients dropped
> silently by gmime.

Well, it's not this bad, I only lose the rest of the display name and
the true email address for the recipients where this matches.  Later
recipients are preserved.  That's better than I thought, but definitely
not good, since the same email list can have multiple display names
depending on the sender's preferences, and now I have no guarantee that
notmuch will have the true email address indexed.  Hopefully this poor
behavior is related to my exposure to Exchange and isn't contagious
without willful stupidity. :)

This behavior also breaks the idea that I can just copy and paste from
the To: field into a search, since some terms will be missing.

It looks like it would be better to stuff the entire string of the To:
field directly into Xapian.  GMime will give you a string output of what it
figured out from the message header, but that already has terms pruned
as shown below:

>From the raw file:

To: One Big Happy , 
dist.Happy Group ,
This Really Stinks ,
This.WillPrune ,
This Will Not Prune 

Trace output:

Email address list: One Big Happy , dist.Happy, This 
Really Stinks , This.WillPrune, This Will 
Not Prune 
Email address: One Big Happy 
Email address: dist.Happy
Email address: This Really Stinks 
Email address: This.WillPrune
Email address: This Will Not Prune 

Any suggestions for how to fix this?  Or is my mail broken irreparably?

-Mark



Debugging strangeness in To: field

2011-04-06 Thread Aaron Williamson
Hi Mark,

On 04/06/2011 02:58 PM, Mark Anderson wrote:
> Do you have any hints about how I could figure out why gmime doesn't
> like this To: list?
> 
> To: One Big Happy , dist.Happy Group
>   

I may be way off, but I wonder if it's seeing "dist.Happy" and confusing it for
an email address (or at least a domain) rather than the display name for the
email address.  Maybe display names of the form xxx.yyy need quotes?

Best,
Aaron


Re: Debugging strangeness in To: field

2011-04-06 Thread Mark Anderson
On Wed, 06 Apr 2011 13:35:44 -0600, Mark Anderson  
wrote:
Hello All,

> It is rather painful that I can have a lot of recipients dropped
> silently by gmime.

Well, it's not this bad, I only lose the rest of the display name and
the true email address for the recipients where this matches.  Later
recipients are preserved.  That's better than I thought, but definitely
not good, since the same email list can have multiple display names
depending on the sender's preferences, and now I have no guarantee that
notmuch will have the true email address indexed.  Hopefully this poor
behavior is related to my exposure to Exchange and isn't contagious
without willful stupidity. :)

This behavior also breaks the idea that I can just copy and paste from
the To: field into a search, since some terms will be missing.

It looks like it would be better to stuff the entire string of the To:
field directly into Xapian.  GMime will give you a string output of what it
figured out from the message header, but that already has terms pruned
as shown below:

>From the raw file:

To: One Big Happy , 
dist.Happy Group ,
This Really Stinks ,
This.WillPrune ,
This Will Not Prune 

Trace output:

Email address list: One Big Happy , dist.Happy, This 
Really Stinks , This.WillPrune, This Will Not 
Prune 
Email address: One Big Happy 
Email address: dist.Happy
Email address: This Really Stinks 
Email address: This.WillPrune
Email address: This Will Not Prune 

Any suggestions for how to fix this?  Or is my mail broken irreparably?

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Debugging strangeness in To: field

2011-04-06 Thread Mark Anderson
On Wed, 6 Apr 2011 14:10:51 -0500, Aaron Williamson  wrote:
> Hi Mark,
> 
> On 04/06/2011 02:58 PM, Mark Anderson wrote:
> > Do you have any hints about how I could figure out why gmime doesn't
> > like this To: list?
> > 
> > To: One Big Happy , dist.Happy Group
> > 
> 
> I may be way off, but I wonder if it's seeing "dist.Happy" and confusing it 
> for
> an email address (or at least a domain) rather than the display name for the
> email address.  Maybe display names of the form xxx.yyy need quotes?

Hi Aaron,

H, if that's the case, then perhaps I need to process my headers to
"fix" Exchange email addresses with display names containing '.'

I'm fairly certain that Exchange "display names" for internal email
lists is the source, since instrumenting my notmuch, I see other
instances of the same craziness with incompletely indexed "To:" lists.

Arg, I'm reading RFC #822, and it seems pretty clear that originally
this would not be allowed.  '.' is a special character and wouldn't have
been allowed unquoted in the display name.

I could hope that some leeway has been added in more recent RFC's, but
with MSoft's traditional implement first, specs later methodology, it
seems unlikely.

What a bother.

It is rather painful that I can have a lot of recipients dropped
silently by gmime.

Thanks,
-Mark

> 
> Best,
> Aaron
> 



Debugging strangeness in To: field

2011-04-06 Thread Mark Anderson
Hello all,

Do you have any hints about how I could figure out why gmime doesn't
like this To: list?

To: One Big Happy , dist.Happy Group


I added printouts to lib/index.cc just so I could try to figure out what
was missing, and I see this:

Email address list: One Big Happy , dist.Happy
Email address: One Big Happy 
Email address: dist.Happy

Why did it do that?  I will try pulling a new gmime source to see if
this is a fixed bug, but I don't know the RFC's well enough to know if
there is any wrapping behavior that justifies gmime in considering the
email list complete.

Any hints are appreciated.

Thanks,
-Mark



Re: Debugging strangeness in To: field

2011-04-06 Thread Mark Anderson
On Wed, 6 Apr 2011 14:10:51 -0500, Aaron Williamson  
wrote:
> Hi Mark,
> 
> On 04/06/2011 02:58 PM, Mark Anderson wrote:
> > Do you have any hints about how I could figure out why gmime doesn't
> > like this To: list?
> > 
> > To: One Big Happy , dist.Happy Group
> > 
> 
> I may be way off, but I wonder if it's seeing "dist.Happy" and confusing it 
> for
> an email address (or at least a domain) rather than the display name for the
> email address.  Maybe display names of the form xxx.yyy need quotes?

Hi Aaron,

H, if that's the case, then perhaps I need to process my headers to
"fix" Exchange email addresses with display names containing '.'

I'm fairly certain that Exchange "display names" for internal email
lists is the source, since instrumenting my notmuch, I see other
instances of the same craziness with incompletely indexed "To:" lists.

Arg, I'm reading RFC #822, and it seems pretty clear that originally
this would not be allowed.  '.' is a special character and wouldn't have
been allowed unquoted in the display name.

I could hope that some leeway has been added in more recent RFC's, but
with MSoft's traditional implement first, specs later methodology, it
seems unlikely.

What a bother.

It is rather painful that I can have a lot of recipients dropped
silently by gmime.

Thanks,
-Mark

> 
> Best,
> Aaron
> 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Debugging strangeness in To: field

2011-04-06 Thread Aaron Williamson
Hi Mark,

On 04/06/2011 02:58 PM, Mark Anderson wrote:
> Do you have any hints about how I could figure out why gmime doesn't
> like this To: list?
> 
> To: One Big Happy , dist.Happy Group
>   

I may be way off, but I wonder if it's seeing "dist.Happy" and confusing it for
an email address (or at least a domain) rather than the display name for the
email address.  Maybe display names of the form xxx.yyy need quotes?

Best,
Aaron
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Debugging strangeness in To: field

2011-04-06 Thread Mark Anderson
Hello all,

Do you have any hints about how I could figure out why gmime doesn't
like this To: list?

To: One Big Happy , dist.Happy Group


I added printouts to lib/index.cc just so I could try to figure out what
was missing, and I see this:

Email address list: One Big Happy , dist.Happy
Email address: One Big Happy 
Email address: dist.Happy

Why did it do that?  I will try pulling a new gmime source to see if
this is a fixed bug, but I don't know the RFC's well enough to know if
there is any wrapping behavior that justifies gmime in considering the
email list complete.

Any hints are appreciated.

Thanks,
-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch