Re: duplicate results in aoxexport

[Mark Felder]

On Thu, May 11, 2017, at 18:36, Carlos Hanson wrote:
> Greetings,
> 
> I just started playing with Archiveopteryx. I have just a few emails in
> the
> database. When I use aoxexport, I receive duplicates of the same email.
> Is
> this expected? I did not find a man page for aoxexport.
> 
> For my most recent test, I searched for the message-id using header then
> grepped the line  staring with "From ", since that is the beginning of
> each
> message:
> 
> $ aoxexport header verylon...@mail.gmail.com | grep '^From '
> 
> >From user@email  Thu May 11 22:58:56 2017
> >From user@email  Thu May 11 22:58:56 2017
> >From user@email  Thu May 11 22:58:56 2017
> 
> 
> I checked my mail.log and only see one instance of *Archiveopteryx:
> 5910/5/9/5/1 Injecting message  >*.
> 
> Thanks. I'm looking forward to playing with this more.
> 
> 

Did that email come from GMail and have multiple labels on it? If so, to
aox it might look like multiple emails stored in different folders. It
should be deduplicated within aox, but I could understand why it would
output multiple copies with aox export.

-- 
  Mark Felder
  f...@feld.me

Re: duplicate results in aoxexport

[Arnt Gulbrandsen]

Carlos Hanson writes:
It's coming from GMail via routing through SMTP. It is not 
using IMAP to check a mailbox. I don't think labels are 
available during the SMTP process.


No, but an import/export process from gmail can/will result in multiple 
"copies" of a message in aoxexport. You start with one copy in gmail, then 
there are multiple copies during the transfer, which are mostly merged 
during the aox injection but aox still remembers that there were multiple 
copies injected, then the merge is effectively undone by aoxexport.


What is the concrete problem?

Arnt

Re: duplicate results in aoxexport

[Carlos Hanson]

I would like this to be a mail archive which can be searched when
necessary. ​If I perform a search and multiple emails match, I expect to
receive one copy of each matched message.​ I need to either figure out what
I am doing wrong when I get duplicate messages or change my expectations
and determine a way to remove duplicate results.



Carlos Hanson
Tigard-Tualatin School District
Web and System Administrator
503.431.4053

On Mon, May 15, 2017 at 2:17 PM, Arnt Gulbrandsen 
wrote:

> Carlos Hanson writes:
>
>> It's coming from GMail via routing through SMTP. It is not using IMAP to
>> check a mailbox. I don't think labels are available during the SMTP process.
>>
>
> No, but an import/export process from gmail can/will result in multiple
> "copies" of a message in aoxexport. You start with one copy in gmail, then
> there are multiple copies during the transfer, which are mostly merged
> during the aox injection but aox still remembers that there were multiple
> copies injected, then the merge is effectively undone by aoxexport.
>
> What is the concrete problem?
>
> Arnt
>
>

Re: duplicate results in aoxexport

[Carlos Hanson]

It's coming from GMail via routing through SMTP. It is not using IMAP to
check a mailbox. I don't think labels are available during the SMTP process.


Carlos Hanson
Tigard-Tualatin School District
Web and System Administrator
503.431.4053

On Mon, May 15, 2017 at 2:01 PM, Mark Felder  wrote:

>
>
> On Thu, May 11, 2017, at 18:36, Carlos Hanson wrote:
> > Greetings,
> >
> > I just started playing with Archiveopteryx. I have just a few emails in
> > the
> > database. When I use aoxexport, I receive duplicates of the same email.
> > Is
> > this expected? I did not find a man page for aoxexport.
> >
> > For my most recent test, I searched for the message-id using header then
> > grepped the line  staring with "From ", since that is the beginning of
> > each
> > message:
> >
> > $ aoxexport header verylon...@mail.gmail.com | grep '^From '
> >
> > >From user@email  Thu May 11 22:58:56 2017
> > >From user@email  Thu May 11 22:58:56 2017
> > >From user@email  Thu May 11 22:58:56 2017
> >
> >
> > I checked my mail.log and only see one instance of *Archiveopteryx:
> > 5910/5/9/5/1 Injecting message  > >*.
> >
> > Thanks. I'm looking forward to playing with this more.
> >
> >
>
> Did that email come from GMail and have multiple labels on it? If so, to
> aox it might look like multiple emails stored in different folders. It
> should be deduplicated within aox, but I could understand why it would
> output multiple copies with aox export.
>
> --
>   Mark Felder
>   f...@feld.me
>