Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Matteo Beccati
On 01/02/2010 03:27, Alvaro Herrera wrote: Matteo Beccati wrote: Incidentally, I've just found out that the mailing lists are dropping some messages. According to my qmail logs the AOX account never received Joe's message yesterday, nor quite a few others: M156252, M156259, M156262, M156273,

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Magnus Hagander
2010/2/1 Matteo Beccati p...@beccati.com: On 01/02/2010 03:27, Alvaro Herrera wrote: Matteo Beccati wrote: Incidentally, I've just found out that the mailing lists are dropping some messages. According to my qmail logs the AOX account never received Joe's message yesterday, nor quite a few

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Matteo Beccati
On 01/02/2010 10:26, Magnus Hagander wrote: Does the MBOX importer support incremental loading? Because majordomo spits out MBOX files for us already. Unfortunately the aoximport shell command doesn't support incremental loading. One option could be to use SMTP with a subscription as the

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Magnus Hagander
2010/2/1 Matteo Beccati p...@beccati.com: On 01/02/2010 10:26, Magnus Hagander wrote: Does the MBOX importer support incremental loading? Because majordomo spits out MBOX files for us already. Unfortunately the aoximport shell command doesn't support incremental loading. One option could

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Matteo Beccati
On 01/02/2010 15:03, Magnus Hagander wrote: 2010/2/1 Matteo Beccatip...@beccati.com: My main concern is that we'd need to overcomplicate the thread detection algorithm so that it better deals with delayed messages: as it currently works, the replies to a missing message get linked to the

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Magnus Hagander
2010/2/1 Matteo Beccati p...@beccati.com: On 01/02/2010 15:03, Magnus Hagander wrote: 2010/2/1 Matteo Beccatip...@beccati.com: My main concern is that we'd need to overcomplicate the thread detection algorithm so that it better deals with delayed messages: as it currently works, the

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Tom Lane
Matteo Beccati p...@beccati.com writes: My main concern is that we'd need to overcomplicate the thread detection algorithm so that it better deals with delayed messages: as it currently works, the replies to a missing message get linked to the grand-parent. Injecting the missing message

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Robert Haas
On Mon, Feb 1, 2010 at 11:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Matteo Beccati p...@beccati.com writes: My main concern is that we'd need to overcomplicate the thread detection algorithm so that it better deals with delayed messages: as it currently works, the replies to a missing message

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Robert Haas
On Mon, Feb 1, 2010 at 11:41 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Feb 1, 2010 at 11:28 AM, Tom Lane t...@sss.pgh.pa.us wrote:        * A sends a message        * B replies, cc'ing A and the list        * B's reply to list is delayed by

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Feb 1, 2010 at 11:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: * A sends a message * B replies, cc'ing A and the list * B's reply to list is delayed by greylisting * A replies to B's reply (cc'ing list) * A's

Re: [HACKERS] mailing list archiver chewing patches

2010-02-01 Thread Magnus Hagander
2010/2/1 Robert Haas robertmh...@gmail.com: On Mon, Feb 1, 2010 at 11:41 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Feb 1, 2010 at 11:28 AM, Tom Lane t...@sss.pgh.pa.us wrote:        * A sends a message        * B replies, cc'ing A and the list  

Re: [HACKERS] mailing list archiver chewing patches

2010-01-31 Thread Matteo Beccati
On 30/01/2010 22:18, Joe Conway wrote: On 01/30/2010 01:14 PM, Dimitri Fontaine wrote: Matteo Beccatip...@beccati.com writes: I've been following the various suggestions. Please take a look at the updated archives proof of concept: http://archives.beccati.org/ I like the features a lot,

Re: [HACKERS] mailing list archiver chewing patches

2010-01-31 Thread Magnus Hagander
On Sat, Jan 30, 2010 at 22:43, Matteo Beccati p...@beccati.com wrote: On 30/01/2010 17:54, Alvaro Herrera wrote: * While I don't personally care, some are going to insist that the site works with Javascript disabled.  I didn't try but from your description it doesn't seem like it would.  Is

Re: [HACKERS] mailing list archiver chewing patches

2010-01-31 Thread Matteo Beccati
On 31/01/2010 13:45, Magnus Hagander wrote: On Sat, Jan 30, 2010 at 22:43, Matteo Beccatip...@beccati.com wrote: On 30/01/2010 17:54, Alvaro Herrera wrote: * While I don't personally care, some are going to insist that the site works with Javascript disabled. I didn't try but from your

Re: [HACKERS] mailing list archiver chewing patches

2010-01-31 Thread Magnus Hagander
On Sun, Jan 31, 2010 at 15:09, Matteo Beccati p...@beccati.com wrote: On 31/01/2010 13:45, Magnus Hagander wrote: On Sat, Jan 30, 2010 at 22:43, Matteo Beccatip...@beccati.com  wrote: On 30/01/2010 17:54, Alvaro Herrera wrote: * While I don't personally care, some are going to insist that

Re: [HACKERS] mailing list archiver chewing patches

2010-01-31 Thread Alvaro Herrera
Matteo Beccati wrote: Incidentally, I've just found out that the mailing lists are dropping some messages. According to my qmail logs the AOX account never received Joe's message yesterday, nor quite a few others: M156252, M156259, M156262, M156273, M156275 and I've verified that it also

Re: [HACKERS] mailing list archiver chewing patches

2010-01-30 Thread Matteo Beccati
Il 19/01/2010 09:44, Magnus Hagander ha scritto: As long as the templating is separated from the code, it doesn't matter if it's a dedicated templating engine or PHP. The point being, focus on the contents and interface, porting the actual HTML-generation is likely to be easy compared to that.

Re: [HACKERS] mailing list archiver chewing patches

2010-01-30 Thread Alvaro Herrera
Matteo Beccati wrote: Il 19/01/2010 09:44, Magnus Hagander ha scritto: As long as the templating is separated from the code, it doesn't matter if it's a dedicated templating engine or PHP. The point being, focus on the contents and interface, porting the actual HTML-generation is likely to be

Re: [HACKERS] mailing list archiver chewing patches

2010-01-30 Thread Dimitri Fontaine
Matteo Beccati p...@beccati.com writes: I've been following the various suggestions. Please take a look at the updated archives proof of concept: http://archives.beccati.org/ I like the features a lot, and the only remarks I can think about are bikeschedding, so I'll let it to the web team

Re: [HACKERS] mailing list archiver chewing patches

2010-01-30 Thread Joe Conway
On 01/30/2010 01:14 PM, Dimitri Fontaine wrote: Matteo Beccati p...@beccati.com writes: I've been following the various suggestions. Please take a look at the updated archives proof of concept: http://archives.beccati.org/ I like the features a lot, and the only remarks I can think about

Re: [HACKERS] mailing list archiver chewing patches

2010-01-30 Thread Matteo Beccati
On 30/01/2010 17:54, Alvaro Herrera wrote: Matteo Beccati wrote: Il 19/01/2010 09:44, Magnus Hagander ha scritto: As long as the templating is separated from the code, it doesn't matter if it's a dedicated templating engine or PHP. The point being, focus on the contents and interface, porting

Re: [HACKERS] mailing list archiver chewing patches

2010-01-19 Thread Matteo Beccati
Il 18/01/2010 18:42, Magnus Hagander ha scritto: On Mon, Jan 18, 2010 at 18:31, Matteo Beccatip...@beccati.com wrote: Il 18/01/2010 15:55, Magnus Hagander ha scritto: If it wasn't for the fact that we're knee deep in two other major projects for the infrastructure team right now, I'd be all

Re: [HACKERS] mailing list archiver chewing patches

2010-01-19 Thread Magnus Hagander
On Tue, Jan 19, 2010 at 09:11, Matteo Beccati p...@beccati.com wrote: Il 18/01/2010 18:42, Magnus Hagander ha scritto: On Mon, Jan 18, 2010 at 18:31, Matteo Beccatip...@beccati.com  wrote: Il 18/01/2010 15:55, Magnus Hagander ha scritto: If it wasn't for the fact that we're knee deep in two

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Matteo Beccati
Il 16/01/2010 14:21, Matteo Beccati ha scritto: Il 16/01/2010 11:48, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: Anyway, I've made further changes and I would say that at this point the PoC is feature complete. There surely are still some rough edges and a few things to

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Magnus Hagander
2010/1/18 Matteo Beccati p...@beccati.com: Il 16/01/2010 14:21, Matteo Beccati ha scritto: Il 16/01/2010 11:48, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: Anyway, I've made further changes and I would say that at this point the PoC is feature complete. There

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: Also, I tink one of the main issues with the archives today that people bring up is the inability to have threads cross months. I think that should be fixed. Basically, get rid of the grouping by month for a more dynamic way to browse. Clic a mail

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Alvaro Herrera
Magnus Hagander wrote: 2010/1/18 Matteo Beccati p...@beccati.com: My question now is... what next? :) Gee, I disappear for a week and look what happens -- we get streaming replication, a revamped archives site, and maybe something else that I haven't seen yet. I love it :-) If it wasn't

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Matteo Beccati
Il 18/01/2010 15:55, Magnus Hagander ha scritto: If it wasn't for the fact that we're knee deep in two other major projects for the infrastructure team right now, I'd be all over this :-) But we really need to complete that before we put anything new in production here. Sure, that's completely

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Matteo Beccati
Il 18/01/2010 16:19, Dimitri Fontaine ha scritto: Magnus Hagandermag...@hagander.net writes: Also, I tink one of the main issues with the archives today that people bring up is the inability to have threads cross months. I think that should be fixed. Basically, get rid of the grouping by month

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Magnus Hagander
On Mon, Jan 18, 2010 at 18:31, Matteo Beccati p...@beccati.com wrote: Il 18/01/2010 15:55, Magnus Hagander ha scritto: If it wasn't for the fact that we're knee deep in two other major projects for the infrastructure team right now, I'd be all over this :-) But we really need to complete that

Re: [HACKERS] mailing list archiver chewing patches

2010-01-18 Thread Magnus Hagander
On Mon, Jan 18, 2010 at 18:35, Matteo Beccati p...@beccati.com wrote: Il 18/01/2010 16:19, Dimitri Fontaine ha scritto: Magnus Hagandermag...@hagander.net  writes: Also, I tink one of the main issues with the archives today that people bring up is the inability to have threads cross months.

Re: [HACKERS] mailing list archiver chewing patches

2010-01-16 Thread Dimitri Fontaine
Matteo Beccati p...@beccati.com writes: Anyway, I've made further changes and I would say that at this point the PoC is feature complete. There surely are still some rough edges and a few things to clean up, but I'd like to get your feedback once again: http://archives.beccati.org I've been

Re: [HACKERS] mailing list archiver chewing patches

2010-01-16 Thread Matteo Beccati
Il 16/01/2010 11:48, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: Anyway, I've made further changes and I would say that at this point the PoC is feature complete. There surely are still some rough edges and a few things to clean up, but I'd like to get your feedback

Re: [HACKERS] mailing list archiver chewing patches

2010-01-15 Thread Matteo Beccati
Hi everyone, Il 14/01/2010 19:36, David Fetter ha scritto: On Thu, Jan 14, 2010 at 03:08:22PM +0100, Matteo Beccati wrote: Il 14/01/2010 14:39, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: Any improvements to sorting are welcome :) ... ARRAY[uid] ...

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Matteo Beccati
Il 14/01/2010 08:22, Matteo Beccati ha scritto: Hi, 3) A nice set of SQL queries to return message, parts, threads, folders based on $criteria (search, id, folder, etc) I guess Matteo's working on that… Right, but this is where I want to see the AOX schema imporove... In ways like adding

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Dimitri Fontaine
Matteo Beccati p...@beccati.com writes: I've extended AOX with a trigger that takes care of filling a separate table that's used to display the index pages. The new table also stores threading information (standard headers + Exchange headers support) and whether or not the email has

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Dave Page
On Thu, Jan 14, 2010 at 7:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Matteo Beccati p...@beccati.com writes: I've extended AOX with a trigger that takes care of filling a separate table that's used to display the index pages. The new table also stores threading information (standard

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Matteo Beccati
Il 14/01/2010 14:39, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: I've extended AOX with a trigger that takes care of filling a separate table that's used to display the index pages. The new table also stores threading information (standard headers + Exchange headers

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Matteo Beccati
Il 14/01/2010 14:46, Dave Page ha scritto: On Thu, Jan 14, 2010 at 7:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Matteo Beccatip...@beccati.com writes: I've extended AOX with a trigger that takes care of filling a separate table that's used to display the index pages. The new table

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Dave Page
On Thu, Jan 14, 2010 at 8:02 PM, Matteo Beccati p...@beccati.com wrote: Il 14/01/2010 14:46, Dave Page ha scritto: On Thu, Jan 14, 2010 at 7:09 PM, Dimitri Fontaine dfonta...@hi-media.com  wrote: Matteo Beccatip...@beccati.com  writes: I've extended AOX with a trigger that takes care of

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Dimitri Fontaine
Matteo Beccati p...@beccati.com writes: WITH RECURSIVE t (mailbox, uid, date, subject, sender, has_attachments, parent_uid, idx, depth) AS ( SELECT mailbox, uid, date, subject, sender, has_attachments, parent_uid, uid::text, 1 FROM arc_messages WHERE parent_uid IS NULL AND mailbox = 15

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Matteo Beccati
Il 14/01/2010 15:47, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: WITH RECURSIVE t (mailbox, uid, date, subject, sender, has_attachments, parent_uid, idx, depth) AS ( SELECT mailbox, uid, date, subject, sender, has_attachments, parent_uid, uid::text, 1 FROM

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread Magnus Hagander
On Thu, Jan 14, 2010 at 16:06, Matteo Beccati p...@beccati.com wrote: Il 14/01/2010 15:47, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com  writes: WITH RECURSIVE t (mailbox, uid, date, subject, sender, has_attachments, parent_uid, idx, depth) AS (   SELECT mailbox, uid, date,

Re: [HACKERS] mailing list archiver chewing patches

2010-01-14 Thread David Fetter
On Thu, Jan 14, 2010 at 03:08:22PM +0100, Matteo Beccati wrote: Il 14/01/2010 14:39, Dimitri Fontaine ha scritto: Matteo Beccatip...@beccati.com writes: I've extended AOX with a trigger that takes care of filling a separate table that's used to display the index pages. The new table also

Re: [HACKERS] mailing list archiver chewing patches

2010-01-13 Thread Matteo Beccati
Hi, 3) A nice set of SQL queries to return message, parts, threads, folders based on $criteria (search, id, folder, etc) I guess Matteo's working on that… Right, but this is where I want to see the AOX schema imporove... In ways like adding persistant tables for threading, which are

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Dimitri Fontaine
Dave Page dp...@pgadmin.org writes: 2010/1/12 Matteo Beccati p...@beccati.com: So, I've decided to spend a bit more time on this and here is a proof of concept web app that displays mailing list archives reading from the AOX database: http://archives.beccati.org/ Seems to work. Hehe,

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Magnus Hagander
On Tue, Jan 12, 2010 at 10:05, Dimitri Fontaine dfonta...@hi-media.com wrote: Dave Page dp...@pgadmin.org writes: 2010/1/12 Matteo Beccati p...@beccati.com: So, I've decided to spend a bit more time on this and here is a proof of concept web app that displays mailing list archives reading

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Magnus Hagander
On Mon, Jan 11, 2010 at 15:00, Abhijit Menon-Sen a...@toroid.org wrote: (Many thanks to Dimitri for bringing this thread to my attention.) At 2010-01-11 10:46:10 +0100, mag...@hagander.net wrote: As for AOX, my understanding is that it is no longer maintained, so I'd be worried about

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Joshua D. Drake
On Tue, 2010-01-12 at 10:24 +0530, Dave Page wrote: 2010/1/12 Matteo Beccati p...@beccati.com: So, I've decided to spend a bit more time on this and here is a proof of concept web app that displays mailing list archives reading from the AOX database: http://archives.beccati.org/

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On Tue, 2010-01-12 at 10:24 +0530, Dave Page wrote: So just to put this into perspective and give anyone paying attention an idea of the pain that lies ahead should they decide to work on this: - We need to import the old archives (of which

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Joshua D. Drake
On Tue, 2010-01-12 at 11:54 -0500, Tom Lane wrote: Or, we just leave the current infrastructure in place and use a new one for all new messages going forward. We shouldn't limit our ability to have a decent system due to decisions of the past. -1. What's the point of having archives?

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Dave Page
On Tue, Jan 12, 2010 at 10:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Joshua D. Drake j...@commandprompt.com writes: On Tue, 2010-01-12 at 10:24 +0530, Dave Page wrote: So just to put this into perspective and give anyone paying attention an idea of the pain that lies ahead should they decide

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Magnus Hagander
On Tue, Jan 12, 2010 at 18:34, Dave Page dp...@pgadmin.org wrote: On Tue, Jan 12, 2010 at 10:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Joshua D. Drake j...@commandprompt.com writes: On Tue, 2010-01-12 at 10:24 +0530, Dave Page wrote: So just to put this into perspective and give anyone paying

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Matteo Beccati
Il 12/01/2010 10:30, Magnus Hagander ha scritto: The problem is usually with strange looking emails with 15 different MIME types. If we can figure out the proper way to render that, the rest really is just a SMOP. Yeah, I was expecting some, but all the message I've looked at seemed to be

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Matteo Beccati
Il 12/01/2010 19:54, Magnus Hagander ha scritto: On Tue, Jan 12, 2010 at 18:34, Dave Pagedp...@pgadmin.org wrote: On Tue, Jan 12, 2010 at 10:24 PM, Tom Lanet...@sss.pgh.pa.us wrote: Joshua D. Drakej...@commandprompt.com writes: On Tue, 2010-01-12 at 10:24 +0530, Dave Page wrote: So just

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Magnus Hagander
On Tue, Jan 12, 2010 at 20:56, Matteo Beccati p...@beccati.com wrote: Il 12/01/2010 10:30, Magnus Hagander ha scritto: The problem is usually with strange looking emails with 15 different MIME types. If we can figure out the proper way to render that, the rest really is just a SMOP. Yeah, I

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Aidan Van Dyk
I'll note that the whole idea of a email archive interface might be a very good advocacy project as well. AOX might not be a perfect fit, but it could be a good learning experience... Really, all the PG mail archives need is: 1) A nice normalized DB schema representing mail messages and their

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Matteo Beccati
Il 12/01/2010 21:04, Magnus Hagander ha scritto: On Tue, Jan 12, 2010 at 20:56, Matteo Beccatip...@beccati.com wrote: Il 12/01/2010 10:30, Magnus Hagander ha scritto: The problem is usually with strange looking emails with 15 different MIME types. If we can figure out the proper way to

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Dimitri Fontaine
Aidan Van Dyk ai...@highrise.ca writes: I'll note that the whole idea of a email archive interface might be a very good advocacy project as well. AOX might not be a perfect fit, but it could be a good learning experience... Really, all the PG mail archives need is: 1) A nice normalized DB

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Aidan Van Dyk
* Dimitri Fontaine dfonta...@hi-media.com [100112 16:28]: 1) A nice normalized DB schema representing mail messages and their relations to other message and recipients (or folders) We're now hoping that this one will fit: http://www.archiveopteryx.org/schema Yup, and it provides a

Re: [HACKERS] mailing list archiver chewing patches

2010-01-12 Thread Dimitri Fontaine
Aidan Van Dyk ai...@highrise.ca writes: aox has that either as a bulk importer or as a MDA. Yup, LMTP is ideally suited for that too. Yes. 3) A nice set of SQL queries to return message, parts, threads, folders based on $criteria (search, id, folder, etc) I guess Matteo's working on

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Magnus Hagander
2010/1/11 Alvaro Herrera alvhe...@commandprompt.com: Dimitri Fontaine wrote: Andrew Dunstan and...@dunslane.net writes: That is assuming that the MUA gives you the option of specifying the attachment MIME type. Many (including mine) do not. It would mean an extra step - I'd have to gzip

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@commandprompt.com writes: Absolutely. The month boundary problem boils down to the fact that Mhonarc does not scale very well, so we can't have mboxes that are too large. This is why most people split their archives per month, and then each month is published as an

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Matteo Beccati
Hi, Il 11/01/2010 11:18, Dimitri Fontaine ha scritto: AOX is already a database backed email solution, offering an archive page with searching. I believe the searching is baked by tsearch indexing. That's why I think it'd be suitable. They already archive and offer search over one of our

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Dave Page
On Mon, Jan 11, 2010 at 5:23 PM, Matteo Beccati p...@beccati.com wrote: Hi, Il 11/01/2010 11:18, Dimitri Fontaine ha scritto: AOX is already a database backed email solution, offering an archive page with searching. I believe the searching is baked by tsearch indexing. That's why I think

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Matteo Beccati
Il 11/01/2010 12:58, Dave Page ha scritto: On Mon, Jan 11, 2010 at 5:23 PM, Matteo Beccatip...@beccati.com wrote: I recall having tried AOX a long time ago but I can't remember the reason why I was not satisfied. I guess I can give another try by setting up a test ML archive. I tried it too,

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: As for AOX, my understanding is that it is no longer maintained, so I'd be worried about choosing such a solution for a complex problem. But it's open for discussion. Ouch. -- dim -- Sent via pgsql-hackers mailing list

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Dimitri Fontaine
Dave Page dp...@pgadmin.org writes: I recall having tried AOX a long time ago but I can't remember the reason why I was not satisfied. I guess I can give another try by setting up a test ML archive. I tried it too, before I started writing the new prototype archiver from scratch. I too

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Magnus Hagander
2010/1/11 Dimitri Fontaine dfonta...@hi-media.com: Dave Page dp...@pgadmin.org writes: I recall having tried AOX a long time ago but I can't remember the reason why I was not satisfied. I guess I can give another try by setting up a test ML archive. I tried it too, before I started writing

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Dimitri Fontaine
Dimitri Fontaine dfonta...@hi-media.com writes: Magnus Hagander mag...@hagander.net writes: As for AOX, my understanding is that it is no longer maintained, so I'd be worried about choosing such a solution for a complex problem. But it's open for discussion. Ouch. It seems that the company

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Andrew Dunstan
Magnus Hagander wrote: No, the current archiver is a set of MBOX files that are processed incrementally by mhonarc. (yes, this is why it doesn't scale) *search* is in a postgresql database, but it doesn't contain the entire messages - doesn't have attachments for examples - only the parts

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Abhijit Menon-Sen
(Many thanks to Dimitri for bringing this thread to my attention.) At 2010-01-11 10:46:10 +0100, mag...@hagander.net wrote: As for AOX, my understanding is that it is no longer maintained, so I'd be worried about choosing such a solution for a complex problem. I'll keep this short: Oryx, the

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Matteo Beccati
Il 11/01/2010 15:00, Abhijit Menon-Sen ha scritto: I'll keep this short: Oryx, the company behind Archiveopteryx (aox), is no longer around, but the software is still maintained. The developers (myself included) are still interested in keeping it alive. It's been a while since the last release,

Re: [HACKERS] mailing list archiver chewing patches

2010-01-11 Thread Dave Page
2010/1/12 Matteo Beccati p...@beccati.com: So, I've decided to spend a bit more time on this and here is a proof of concept web app that displays mailing list archives reading from the AOX database: http://archives.beccati.org/ Seems to work. Please take it as an exercise I've made

Re: [HACKERS] mailing list archiver chewing patches

2010-01-10 Thread Dimitri Fontaine
Andrew Dunstan and...@dunslane.net writes: That is assuming that the MUA gives you the option of specifying the attachment MIME type. Many (including mine) do not. It would mean an extra step - I'd have to gzip each patch or something like that. That would be unfortunate,as well as imposing

Re: [HACKERS] mailing list archiver chewing patches

2010-01-10 Thread Alvaro Herrera
Dimitri Fontaine wrote: Andrew Dunstan and...@dunslane.net writes: That is assuming that the MUA gives you the option of specifying the attachment MIME type. Many (including mine) do not. It would mean an extra step - I'd have to gzip each patch or something like that. That would be

Re: [HACKERS] mailing list archiver chewing patches

2010-01-09 Thread Tim Bunce
On Sat, Jan 09, 2010 at 02:17:27AM -0300, Alvaro Herrera wrote: Alvaro Herrera wrote: Andrew Dunstan wrote: Tim Bunce's recent patch has been mangled apparently by the list archives. He sent it as an attachment, and that's how I have it in my mailbox, so why isn't it appearing as

Re: [HACKERS] mailing list archiver chewing patches

2010-01-09 Thread Andrew Dunstan
Tim Bunce wrote: It seems that people wanting to send in a patch have two options: send it as text/(something) so it's readable on the archive web page but not copy-n-paste'able because of wordwrapping, or set it as application/octet-stream so it's downloadable but not readable on the web

Re: [HACKERS] mailing list archiver chewing patches

2010-01-09 Thread Alvaro Herrera
Tim Bunce wrote: Try this http://archives.postgresql.org/msgtxt.php?id=20100108124613.gl2...@timac.local That looks like it dumps the raw message. That'll cause problems for any messages using quoted-printable encoding. I'd hazard a guess it also won't do thing right thing for

Re: [HACKERS] mailing list archiver chewing patches

2010-01-08 Thread Alvaro Herrera
Andrew Dunstan wrote: Tim Bunce's recent patch has been mangled apparently by the list archives. He sent it as an attachment, and that's how I have it in my mailbox, so why isn't it appearing as such in the web archive so that it can be nicely downloaded? See

Re: [HACKERS] mailing list archiver chewing patches

2010-01-08 Thread Alvaro Herrera
Alvaro Herrera wrote: Andrew Dunstan wrote: Tim Bunce's recent patch has been mangled apparently by the list archives. He sent it as an attachment, and that's how I have it in my mailbox, so why isn't it appearing as such in the web archive so that it can be nicely downloaded? See