Re: [Fwd: Re: Questions about IMAP and sequences]

[Chris Garrigues]

> From:  "Dan Harkless" <[EMAIL PROTECTED]>
> Date:  Tue, 12 Sep 2000 14:55:35 -0700
>
> Ralph Corderoy <[EMAIL PROTECTED]> writes:
> > Couldn't nmh develop into having multiple back-ends for different
> > formats and cope with some formats not allowing some operations?  Then
> > anno on IMAP would gracefully fail.

I'm entering this thread real late [due to the fact that I'm on a cable modem
and mhost.com seems to block my traffic, so I had to route around].

I'd also like to see a Maildir backend, and it would be nice if the Maildir 
backend and the imap backend were written to have the same mapping between 
Maildir and imap that courier-imap implements.

Chris

-- 
Chris Garrigues http://www.DeepEddy.Com/~cwg/
virCIO  http://www.virCIO.Com
4314 Avenue C   
Austin, TX  78751-3709  +1 512 374 0500

  My email address is an experiment in SPAM elimination.  For an
  explanation of what we're doing, see http://www.DeepEddy.Com/tms.html 

Nobody ever got fired for buying Microsoft,
  but they could get fired for relying on Microsoft.



 PGP signature

Re: [Fwd: Re: Questions about IMAP and sequences]

[chad]

>From my point of view, IMAP offers two main improvements over POP3.
The second is an extension of the first, and hasn't actually been
realized anywhere as far as I know.

1.) remote storage.  I'd like to be able to get to my mail from
multiple different systems without having to log in to one machine
from everywhere (because of limited access, capability, and
security).  Leaving all of my mail on the IMAP server would let me
accomplish this.

2.) Disconnected IMAP.  This is basically the Holy Grail of modern
email systems.  If you're not familiar with this, take a look at
rfc2060 and
http://asg.web.cmu.edu/cyrus/rfc/draft-ietf-imap-disc-01.html, you
keep your mail on a central server, and periodically retrieve
parts of it, make changes offline (that is, while not connected to
the IMAP server), and then `sync' your changes back to the server
whenever you can.  Disconnected IMAP is pretty tricky.  It's also
incredibly attractive, especially if you find yourself often using
a mixture of, say, PDA's with wireless modem, laptops with
moderate-speed home connection, and multiple disjoint desktops
with high-speed connections.

The ``are we making this too complicated'' question is an excellent
one.  The issue in my mind is that in order to use either one of these
features, I'll want to try to keep my `powerful computer with fast
connection' setup as powerful as possible.  Otherwise, I'd forget
about mail clients at all and just use one of the many webmail
interfaces.

chad

Re: [Fwd: Re: Questions about IMAP and sequences]

[Scott Blachowicz]

On Wed, Sep 13, 2000 at 12:02:52AM -0400, Ken Hornstein wrote:
> >Maybe he's talking about the ability of the UW imapd to access the MH
> >folders you have on the server?  I've used it before (in 'pine' to
> >access a folder "{SYSTEM_NAME}#mh/lists/foo", for instance) to do just
> >that.  These days, I would just ssh in to the box and use nmh
> >directly.
> 
> Was that a feature of UW imapd, or just pine could read mh mailboxes?
> I always thought it was the latter (a feature of the c-client library,
> I think).

But it was able to do it remotely (via a client/server style
connection, not just an NFS mount or some such).  I just don't
remember what the communications mechanism was - I remember being
under the impression that it was the imapd doing it...I don't THINK it
was rsh or anything like that, but it's been a while, so I'm not
positive.  And I don't have an easy way to test it out at the moment.

-- 
Scott Blachowicz

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>What provision in RFC 2060 treats flags as a scarce resource?  I missed that
>in my reading.

It might just be my reading of the specification - there is a fair amount
of text regarding the limits w.r.t. permanent flags.

>I know of no IMAP server in common use which does not support arbitrary
>flags.  The issue is which PERMANENTFLAGS they support; I'm not sure of the
>answer to that question.

Right ... sequences and annotations aren't much good if they aren't
permanent :-)

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[John Reinhagen]

In message <[EMAIL PROTECTED]>, also sprach Ken
 Hornstein:

>>> I don't think they're rich enough to
>>> emulate everything that annotations give you.  
>>
>>Because you can't add multiple ones per message, or because they're of a
>>fixed length, or...?
>
>The IMAP specification treats them as a fairly scarce resource.
>I haven't done a survey, but IMAP servers aren't required to let
>you create new ones not on the "approved" tag list.  I think you
>can have as many as you want, but I'm not sure what the limits
>w.r.t. tags are (they're really supposed to be boolean things).

What provision in RFC 2060 treats flags as a scarce resource?  I missed that
in my reading.

I know of no IMAP server in common use which does not support arbitrary
flags.  The issue is which PERMANENTFLAGS they support; I'm not sure of the
answer to that question.

One can get all the flags associated with messages in a particular mailbox,
so one could easily get all existing sequences for that mailbox; see the
SELECT and EXAMINE commands in the RFC (6.3.1 and 6.3.2 respectively).  The
annoying thing is that the SEARCH command (6.4.4) does not allow searching
by flag, so getting the contents of a sequence is a pain.

JCR
--
Observed by Jeff Cooper:
"Headline in the _Arizona Republic_:
247 POUNDS OF COCAINE SEIZED:  TWO HELD
 (Cheap!  Ten percent is little enough.)"

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>Maybe he's talking about the ability of the UW imapd to access the MH
>folders you have on the server?  I've used it before (in 'pine' to
>access a folder "{SYSTEM_NAME}#mh/lists/foo", for instance) to do just
>that.  These days, I would just ssh in to the box and use nmh
>directly.

Was that a feature of UW imapd, or just pine could read mh mailboxes?
I always thought it was the latter (a feature of the c-client library,
I think).

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Scott Blachowicz]

> >  (2)   When away from your main system, you might want to be able to
> > use IMAP to access your nmh folders from a remote site.
> 
> _This_ would require a special IMAP server; I'm not sure one like
> this exists, and would be a lot of work.  The IMAP model, for good
> or bad, is you always keep everything on the server.

Maybe he's talking about the ability of the UW imapd to access the MH
folders you have on the server?  I've used it before (in 'pine' to
access a folder "{SYSTEM_NAME}#mh/lists/foo", for instance) to do just
that.  These days, I would just ssh in to the box and use nmh
directly.

-- 
[EMAIL PROTECTED]

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>> I don't think they're rich enough to
>> emulate everything that annotations give you.  
>
>Because you can't add multiple ones per message, or because they're of a
>fixed length, or...?

The IMAP specification treats them as a fairly scarce resource.
I haven't done a survey, but IMAP servers aren't required to let
you create new ones not on the "approved" tag list.  I think you
can have as many as you want, but I'm not sure what the limits
w.r.t. tags are (they're really supposed to be boolean things).

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Ken Hornstein <[EMAIL PROTECTED]> writes:
> >Well, anno might be implementable via the IMAP "tags" facility, no?
> 
> Are you thinking about "flags"?  

Yeah, sorry -- I'm just picking up the terminology from this discussion.

> I don't think they're rich enough to
> emulate everything that annotations give you.  

Because you can't add multiple ones per message, or because they're of a
fixed length, or...?

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>If we really wanted to use IMAP, we wouldn't be using 'nmh'.

You're not always given a choice, you know.  What if company-wide
message boards are only available via your IMAP server?

>I can see two possible roles for IMAP for the dedicated 'nmh' user:
>
>  (2)   When away from your main system, you might want to be able to
>   use IMAP to access your nmh folders from a remote site.

_This_ would require a special IMAP server; I'm not sure one like
this exists, and would be a lot of work.  The IMAP model, for good
or bad, is you always keep everything on the server.

Here's my thinking: For whatever reason, I have messages I'd like to
read on my IMAP server.  I already know how to use {n}mh - and I'd
like to combine some of the advantages of IMAP (mailbox mobility)
with some of the advantages of nmh ("toolbox" approach).

The basic tools I think that I use most of the time are: inc, next, prev,
show, comp, repl, folder{s}, refile, scan.  Those are relatively easy.
That would give me a nice set of tools to access my mailbox.  I
might not have all of the features I want, but since not using IMAP
isn't an option, at least some of it works.

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>Well, anno might be implementable via the IMAP "tags" facility, no?

Are you thinking about "flags"?  I don't think they're rich enough to
emulate everything that annotations give you.  I'm not sure there's
any other tagging facility in IMAP.

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Ralph Corderoy <[EMAIL PROTECTED]> writes:
> > > Well, the real crux of the problem is that there are some things
> > > that you simply cannot _do_ within the context of IMAP.  The big
> > > one that comes to mind is annotations (there really isn't a way to
> > > modify messages on the server, from my reading of the
> > > specification).
> > 
> > Ouch, that really bites.  That was one of the first questions that
> > popped into my mind when I first started hearing about IMAP, but I've
> > been too lazy to find out if you could do that or not.  The fact that
> > messages are individual text files is one of the biggest wins of nmh
> > for me.
> 
> Me too.  I reformat crappy emails with lines 120 chars long, delete
> large HTML duplicates that add nothing if I want to keep the mail, etc.
> 
> Couldn't nmh develop into having multiple back-ends for different
> formats and cope with some formats not allowing some operations?  Then
> anno on IMAP would gracefully fail.

Well, anno might be implementable via the IMAP "tags" facility, no?

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Neil W Rickert]

Are we making this too complex?

If we really wanted to use IMAP, we wouldn't be using 'nmh'.

I can see two possible roles for IMAP for the dedicated 'nmh' user:

  (1)   As a pipe, to download messages into your nmh folders.  You
would really be using it much as you use POP3.  You might
want IMAP support either because your ISP does not support
POP3, or because support for IMAP is better.  As an example
of the latter, your ISP might support some sort of encrypted
authentication for IMAP, but only support plain text
passwords sent over the wire for POP3.

  (2)   When away from your main system, you might want to be able to
use IMAP to access your nmh folders from a remote site.

It seems to me that neither of the above require fully featured
support for nmh commands over imap.

 -NWR

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ralph Corderoy]

Hi,

> > Well, the real crux of the problem is that there are some things
> > that you simply cannot _do_ within the context of IMAP.  The big
> > one that comes to mind is annotations (there really isn't a way to
> > modify messages on the server, from my reading of the
> > specification).
> 
> Ouch, that really bites.  That was one of the first questions that
> popped into my mind when I first started hearing about IMAP, but I've
> been too lazy to find out if you could do that or not.  The fact that
> messages are individual text files is one of the biggest wins of nmh
> for me.

Me too.  I reformat crappy emails with lines 120 chars long, delete
large HTML duplicates that add nothing if I want to keep the mail, etc.

Couldn't nmh develop into having multiple back-ends for different
formats and cope with some formats not allowing some operations?  Then
anno on IMAP would gracefully fail.


Ralph.

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>See the APPEND IMAP4rev1 command:
>
>append: http://andrew2.andrew.cmu.edu/rfc/rfc1730.html#sec-6.3.10.

Yeah, I had look at this, but it really doesn't work - you can't
_replace_ an old message, you can only add a new one to a folder.  You
can set a system-defined flag called \Answer that signifies whether or
not you've replied to a message (which I guess helps out on the "-" you
have in the scan listing), but it's a poor substitute.

Also means that things that reorder the mailbox, like "sortm" won't
be possible in the conventional sense.  And "folder -pack" is a no-op
(unless we want to forgo using IMAP message sequence numbers and simply
use UIDs and maintain a local mapping.  Icky).

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Shantonu Sen]

> If there's no way to replace a message on the server with a local version,
> then if nmh does local caching of IMAP messages, modification of those
> messages will definitely be an issue.

See the APPEND IMAP4rev1 command:

append: http://andrew2.andrew.cmu.edu/rfc/rfc1730.html#sec-6.3.10.

It appears that append will add an RFC-822 compliant message to the
current folder. Although this is a poor solution, one could fetch the old
message, munge it, delete the server version, and upload the new one.
I don't know if you can restore the sequencing, though

I think message caching will not be the best distribution of resources
for a first go at IMAP support. once we have a shell of imap interface
commands, we can start doing message-optimization, be it local
caching, etc.

Shantonu

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Ken Hornstein <[EMAIL PROTECTED]> writes:
> >IMO, it's rare because people these days don't think of being able to
> >do it; they're used to GUI mail front-ends that don't allow (?) this
> >kind of thing.
> 
> Use an IMAP client recently?  

Well, he made it clear that he hadn't.  Lots of us nmh folks appear to be a
bit ignorant about the intracacies of IMAP since, well, we use nmh for our
mail.

> "Shared" mailboxes are already part of
> the IMAP specification.  Most reasonable ones deal with them just
> fine.
> 
> >This is why I'm writing my tomes ;-) about not losing nmh flexibility
> >with IMAP, wherever it's reasonable to keep it.
> 
> Well, the real crux of the problem is that there are some things that
> you simply cannot _do_ within the context of IMAP.  The big one that
> comes to mind is annotations (there really isn't a way to modify
> messages on the server, from my reading of the specification).

Ouch, that really bites.  That was one of the first questions that popped
into my mind when I first started hearing about IMAP, but I've been too lazy
to find out if you could do that or not.  The fact that messages are
individual text files is one of the biggest wins of nmh for me.  With all
the crappy mail clients out there that do things like force line wrapping
even if it breaks a long URL, I often find myself reformatting mails.  I
know you can get much of the same advantage by forwarding the mail to
yourself and editing it then, but then the From: is wrong and such (too bad
you can't edit a dist'ed mail).

If there's no way to replace a message on the server with a local version,
then if nmh does local caching of IMAP messages, modification of those
messages will definitely be an issue.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>IMO, it's rare because people these days don't think of being able to
>do it; they're used to GUI mail front-ends that don't allow (?) this
>kind of thing.

Use an IMAP client recently?  "Shared" mailboxes are already part of
the IMAP specification.  Most reasonable ones deal with them just
fine.

>This is why I'm writing my tomes ;-) about not losing nmh flexibility
>with IMAP, wherever it's reasonable to keep it.

Well, the real crux of the problem is that there are some things that
you simply cannot _do_ within the context of IMAP.  The big one that
comes to mind is annotations (there really isn't a way to modify
messages on the server, from my reading of the specification).

What I've been trying to say all along is that all of the essays in the
world won't change the fact that you can't do everything you want to
do with IMAP that you are currently doing with nmh.  So there might
not be a lot of point to writing them.

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Neil W Rickert <[EMAIL PROTECTED]> writes:
> 
> That brings up another point (perhaps a bug?)
> 
>   Msg-Protect: 664
> 
> (from my '.mh_profile') is ignored on "Fcc:".  Thus if a folder
> is shared between users, messages recorded by "Fcc:" are not
> shareable -- they get 600 permissions, which is probably the compiled
> default Msg-Protect).
> 
> Likewise, if 'refile' of a message crosses partition boundaries,
> so that the file must be copied, the refile message does not
> honor Msg-Protect, and does not preserve the original
> message filemode.  It apparently gets the original filemode masked
> by the umask.
> 
> Both of these problems would hinder sharing a folder.  Incidently, MH
> appears to have the same problems, so this is not new to nmh.
> 
> Perhaps the existence of these bugs is evidence that Dan is right in
> his opinion that shared folders are rarely used.  For me the bugs are
> only a minor inconvenience.  I used shared folders, but I share them
> with myself (administrative account and personal account).

Well, considering how hosed the Folder-Protect handling (affecting the
single-user case too) was before I took a whack at it some time back, the
existence of Msg-Protect bugs may not prove *too* much.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Neil W Rickert]

Jerry Peek <[EMAIL PROTECTED]> wrote:
>On 11 September 2000 at 15:01, "Dan Harkless" <[EMAIL PROTECTED]> wrote:
>> multiple users sharing a single nmh folder (with unique sequences) has to be
>> a pretty darn rare situation

>IMO, it's rare because people these days don't think of being able to
>do it; they're used to GUI mail front-ends that don't allow (?) this
>kind of thing.  For instance, I've done that in two different companies
>where I worked in groups that used MH for bug tracking and etc.  We'd
>have central folder(s) of bug reports that everyone accessed (and had
>their own current message, sequences of messages they were working on,
>etc.).  We could use "anno" to make annotations that everyone could see
>(like assigning a bug and tracking its state).  We could use the "mark"
>command to maintain our own sequences (by priority, "do today", etc.).
>"pick" was great for searching bugs and finding out who was working on
>which bugs.  We had a central "bin" directory of scripts to make this
>easier -- and, of course, individual workers could write their own
>scripts.  A central maintainer actually owned the bug folders, inc'ed
>new reports into them, refiled messages into archive folders when a bug
>was fixed, and so on.  (Directory modes were set so only the maintainer
>could move messages out of the folders, but group-write file permissions
>let everyone in the group make annotations on the messages.)  Etc. etc.

That brings up another point (perhaps a bug?)

Msg-Protect: 664

(from my '.mh_profile') is ignored on "Fcc:".  Thus if a folder
is shared between users, messages recorded by "Fcc:" are not
shareable -- they get 600 permissions, which is probably the compiled
default Msg-Protect).

Likewise, if 'refile' of a message crosses partition boundaries,
so that the file must be copied, the refile message does not
honor Msg-Protect, and does not preserve the original
message filemode.  It apparently gets the original filemode masked
by the umask.

Both of these problems would hinder sharing a folder.  Incidently, MH
appears to have the same problems, so this is not new to nmh.

Perhaps the existence of these bugs is evidence that Dan is right in
his opinion that shared folders are rarely used.  For me the bugs are
only a minor inconvenience.  I used shared folders, but I share them
with myself (administrative account and personal account).

 -NWR

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

[Please don't cc mailing list posts to me directly -- I know we're all using
a mail tool that allows arbitrary header editing, so that's no excuse.]

Jerry Peek <[EMAIL PROTECTED]> writes:
> On 11 September 2000 at 15:01, "Dan Harkless" <[EMAIL PROTECTED]> wrote:
> > multiple users sharing a single nmh folder (with unique sequences) has to be
> > a pretty darn rare situation
> 
> IMO, it's rare because people these days don't think of being able to
> do it; they're used to GUI mail front-ends that don't allow (?) this
> kind of thing.  For instance, I've done that in two different companies
> where I worked in groups that used MH for bug tracking and etc.  

Sounds pretty neat.  I do have to admit I have used a shared MH folder
before, as the recipient of a mailing list, and if people had been
interested in following discussions on the list live (as opposed to using it
occasionally as a reference), per-user sequences would've been nice.

I agree that nmh's wealth of features shouldn't be thrown away in the name
of IMAP compatibility, but I also don't think they're necessarily a
legitimate reason why you _can't_ implement IMAP feature X properly.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Jerry Peek]

On 11 September 2000 at 15:01, "Dan Harkless" <[EMAIL PROTECTED]> wrote:
> multiple users sharing a single nmh folder (with unique sequences) has to be
> a pretty darn rare situation

IMO, it's rare because people these days don't think of being able to
do it; they're used to GUI mail front-ends that don't allow (?) this
kind of thing.  For instance, I've done that in two different companies
where I worked in groups that used MH for bug tracking and etc.  We'd
have central folder(s) of bug reports that everyone accessed (and had
their own current message, sequences of messages they were working on,
etc.).  We could use "anno" to make annotations that everyone could see
(like assigning a bug and tracking its state).  We could use the "mark"
command to maintain our own sequences (by priority, "do today", etc.).
"pick" was great for searching bugs and finding out who was working on
which bugs.  We had a central "bin" directory of scripts to make this
easier -- and, of course, individual workers could write their own
scripts.  A central maintainer actually owned the bug folders, inc'ed
new reports into them, refiled messages into archive folders when a bug
was fixed, and so on.  (Directory modes were set so only the maintainer
could move messages out of the folders, but group-write file permissions
let everyone in the group make annotations on the messages.)  Etc. etc.

Sure, there are more sophisticated bug tracking systems that do a lot
more, but that's not my point.  The point is that the flexibility in
nmh makes seamless integration between "email apps" and "other apps"
easy to do.  In GUI environments -- especially ones that glop all the
messages into a single file -- this sort of integration is a lot harder.
This is why I'm writing my tomes ;-) about not losing nmh flexibility
with IMAP, wherever it's reasonable to keep it.

Jerry
-- 
Jerry Peek, [EMAIL PROTECTED], http://www.jpeek.com/

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

clemensF <[EMAIL PROTECTED]> writes:
> > Yeah, your thoughts really make it clear how much potential work is here.
> > Sounds like most of the issues arise from cache handling, though.  Perhaps a
> > first implementation could do everything live on the IMAP server.
> 
> whenever making copies... that's a generalisation of malloc and garbage
> collection.

Even across multiple invocations?  

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Jerry Peek <[EMAIL PROTECTED]> writes:
> In nmh, multiple users can share the same mail folder.  Each user
> typically has their own private context for that shared folder stored
> in their own $HOME/Mail/context file.  So different users can see a
> different view of that folder's sequences.  The same would be true for
> one user who accessed a folder on an IMAP message store from multiple
> hosts: the user would have different context at each host, and that's
> the way it would be.

Um, I'm not sure everyone would be content with "and that's the way it would
be", especially since I believe that's contrary to the spirit of IMAP.  And
multiple users sharing a single nmh folder (with unique sequences) has to be
a pretty darn rare situation, so I'm not sure I buy it as a reason for why
you couldn't implement seamless mailbox viewing from host to host with
nmh/IMAP.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

John Reinhagen <[EMAIL PROTECTED]> writes:
> In message <[EMAIL PROTECTED]>, also sprach "Dan
> Harkless":
> 
> >The last time I remember IMAP support coming up was quite awhile ago, and
> >the commentary (from Richard Coleman??) was that IMAP support probably
> >wouldn't be forthcoming because IMAP would probably spell the eventual death
> >of [n]mh.  I don't recall a lot of posts disagreeing with that view, though
> >I suspect if the subject were brought up again people would disagree.
> 
> I certainly would.  
[...]

Oops.  I see you did send your email to nmh-workers as well.  The duplicate
copy you sent me didn't have a "cc: [EMAIL PROTECTED]" so didn't realize
this.  In any case, no need to send duplicate copies of mailing list posts.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Thanks, John.  I presume your sending this directly to me and not to
nmh-workers was an accident, as no doubt lots of people would be interested
in the expanded description of your nmh/IMAP plans.

John Reinhagen <[EMAIL PROTECTED]> writes:
> In message <[EMAIL PROTECTED]>, also sprach "Dan
> Harkless":
> 
> >The last time I remember IMAP support coming up was quite awhile ago, and
> >the commentary (from Richard Coleman??) was that IMAP support probably
> >wouldn't be forthcoming because IMAP would probably spell the eventual death
> >of [n]mh.  I don't recall a lot of posts disagreeing with that view, though
> >I suspect if the subject were brought up again people would disagree.
> 
> I certainly would.  My guess is that Richard Coleman thought of MH's
> existing mapping between folders and directories/messages and text files to
> be a central feature.  While that is a convenient way to deal with local
> storage, it doesn't seem central to me.  To me, the central feature of [n]mh
> is that it does away with the monolithic mail client environment in favor of
> command- and script-driven environments, and IMAP support wouldn't
> compromise this.  (I admit that this reverses my position as stated on
> comp.mail.mh.)
> 
> >> If not, is there any objection to a Mail::NMH Perl package that provided
> >> such a capability via extension?
> >
> >I don't quite understand the relation between nmh and your proposed Perl
> >package, but I'm sure there'd be no objections to any useful extensions to
> >nmh.
> 
> I'll explain in better detail.  I intend to implement various nmh entities
> -- folders, messages, context (including sequences), profile -- as Perl
> objects.  Any Perl-aware front end could then use and manipulate those
> objects, and one could write a good mail client using such an approach.  (A
> GTK-Perl nmh client would be a nice thing to have.)
> 
> Thanks to OO, I can specify a storage class for each instance of a {Folder,
> Message, Sequence} object, and I can make calls to that storage class to
> fetch, move, etc., the object.  Thus, local objects get treated like classic
> nmh entities, while IMAP objects implement the same functionality through other
> means.
> 
> The advantage of this approach is that it doesn't depend on nmh support for
> IMAP.  The disadvantage is that it might discourage the implementation of
> such support unless some Perl-hater was motivated to correct the
> situation...but then they'd probably just implement the same objects in
> Python/Java/C++/whatever.  nmh would not likely get IMAP support either way.
> 
> If there's an existing intention to put IMAP support in nmh, I'll hold off
> on putting it in my Perl objects; I'd really rather handle it all through
> nmh anyway.  However, if that's not likely to happen, I will design the
> objects to make this possible, and will implement it as time permits.
> 
> JCR
> --
> Observed by Jeff Cooper:
> "Headline in the _Arizona Republic_:
> 247 POUNDS OF COCAINE SEIZED:  TWO HELD
>  (Cheap!  Ten percent is little enough.)"

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Jerry Peek]

On 11 September 2000 at 1:03, Ken Hornstein <[EMAIL PROTECTED]> wrote:
> >I just
> >hope that the implementation can preserve most of the strengths and
> >flexibility of nmh: it's the only email system with all this power,
> >and it'd be sad for the de facto IMAP implementation to hobble nmh.
> 
> Argh.  You've got it BACKWARDS!
> 
> _Not_ having IMAP support is hobbling nmh.

I'm not saying that I think the IMAP support *will* hobble nmh, Ken.
I'm just saying that I hope it doesn't... and I'm giving examples of
places that I think it might have problems.  I'm glad that people who
know IMAP better than *I* do (which, obviously, isn't very well!) are
discussing how to add IMAP support to nmh.

> we should support as many nmh features as possible, but the (IMHO)
> ridiculous desire for 100% MH compatibility has been what has doomed
> previous efforts.

I agree.  When I said "there may not be a perfect one-to-one mapping
between nmh/exmh and IMAP.  Some things may have to bend.", that's just
what I meant.  I was hoping that the folks who do the work know about
the "margins" where nmh can be used -- in places and ways that other
MUAs can't -- so they support as many of those uses as reasonable.
It's good for folks to know what features nmh has, so they can make
*informed* choices about what features to support.  In my experience,
a fair number of people (maybe especially those who've come to nmh
through exmh) don't know all that nmh can do.  I'm trying to help.

Jerry
-- 
Jerry Peek, [EMAIL PROTECTED], http://www.jpeek.com/

Re: [Fwd: Re: Questions about IMAP and sequences]

[Anders Eriksson]

I was thinking along the lines of Shantonu Sen <[EMAIL PROTECTED]> (nmh as 
client) where we take advantage of nmh's powerful command line 
interface and give the core of nmh the ablilty to work on other media 
than raw (nmh) files. It should be possible to define a suitable 
abstraction of a mail in an OO-ish way and implement "plugins" for the 
various storage formats (nmh, imap, etc).

The trick is (which this thread already has discovered) to model stuff 
like sessions/contexts and to make the environment configurable for 
user while keeping .mh_profile sane and not overly bloated with stuff 
from various storage fromats.


As a comment to the caching issue discussed in the thread I'd like to 
see that handled _inside_ the "object" which handles imap storages.

/Anders


Disclaimer, "Object" does not mean C++, you can do OO-like design in C. 
   


> > Anders Eriksson:
> 
> > Are you thinking of an nmh backend to an imap server or nmh ans an imap 
> > client? I'd love to see the actual storage used in nmh 'virtualized', 
> > so we can have the same powerful command line interface and different 
> > storages (mh files, mbx, mbox, imap, ...)
> 
> please elaborate on this.
> 
> clemens
> 



 PGP signature

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>Then there's the question of a "session": doesn't IMAP have the idea
>of "logging on" or "connecting" to an IMAP store for some period of
>time, and preserving the state of that session while the user is
>logged on?

"Not really".  You can have multiple simultaneous connections, and
your clients have to be able to deal with that.

>I wish I knew more about the intricacies of IMAP; I've only used it in
>"clueless mode" ;-) from a GUI front-end and hacked it with JavaMail.
>And sorry if this is obvious; I'm not trying to "talk down" to anyone!
>I'm just trying to share my (almost) 20 years of experiences with MH.
>My "gut feel" is that there may not be a perfect one-to-one mapping
>between nmh/exmh and IMAP.  Some things may have to bend.  I just
>hope that the implementation can preserve most of the strengths and
>flexibility of nmh: it's the only email system with all this power,
>and it'd be sad for the de facto IMAP implementation to hobble nmh.

Argh.  You've got it BACKWARDS!

_Not_ having IMAP support is hobbling nmh.  That's part of the
reason it's being relegated to the dustbin of history (the other
part is that the stalled development at UCI made it difficult to
port to new systems; finally, that's been corrected, but the damaage
lingers on).  When I first met Eric Allman, the very first question
he asked me was, "When the hell is exmh and mh going to support
IMAP?"

I really don't see why not supporting all of the features of nmh
in IMAP is bad; as long as we have something that works, there's
a chance of pulling out of the nmh "death spiral"; it's not like
we're getting a whole lot of new [n]mh users.  When I talk to people
about this, the #1 reason I hear for not using nmh is lack of IMAP
support; instead they're all using pine or mutt or xfmail; MUAs
which all lack the power of nmh, but support IMAP (they're not
exactly given a choice on how to access their mail spools).  Sure,
we should support as many nmh features as possible, but the (IMHO)
ridiculous desire for 100% MH compatibility has been what has doomed
previous efforts.

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>Eww, adding sequences could get complex. Whilst I suppose it might be nice
>for some, do we really need it?

It's actually not that bad.

IMAP has the concept of "message flags", which can be set/read on
a per-message basis.  There's a system one called "\Seen", which
obviously maps to the unseen sequence quite nicely.

You can also manipulate "keywords", which from my reading of the IMAP
specification are simply user-defined flags.  _Depending_ on the IMAP
server implementation and administrative settings, users can create
their own keywords.  Sequences just seem to fall out of that naturally.
I think it's reasonable to only support arbitrary sequences if the
IMAP server supports user-definable keywords.

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>1) Backward compatibility: a new, theoretical version of nmh that uses
>   nmh should be smart enough to use old storage formats if they
>   already exist. If we implement a new storage backend, it should
>   be used only if the user starts using advanced features (like
>   IMAP) that require it. Upgrade and downgrade should be simple.

I, for one, have never advocated using IMAP as the only message store
availble for nmh.  My thinking is very simple: nmh is a collection
of tools for accessing mailboxes.  There's no reason I shouldn't be
able to use the same tools for accessing mailboxes that live on
an IMAP server.

>3) Cache Messages: Should nmh actually download files to cache messages
>   locally?

"no".

>4) Role of `inc': In parallel with 3) is what the role of inc would be.
>   Would it just download an index of new (or unread, depending
>   on implementation) messages? Alternatively, would it download
>   a completely new index to make sure it's completely in sync?

I think "inc" should be relegated to simply display headers of
messages that have the IMAP \Recent flag, and not do anything else.

>   Instead, it might not use the advanced IMAP features and just
>   use the server like a POP machine, and download all the new
>   messages and display a regular scan after it's done.

Since most IMAP servers also implement POP servers, I'm not sure
this is worth bothering.

>6) Refiling: What if a user refiles a message? Does it get immediately
>   updated in the IMAP server,

"yes".

>10) Authentication: We should leave hooks in for varying authentication
>   schemes so we don't need to make lots of changes later. Things
>   that come to mind include IMAP over SSL. Hopefully, the authentication
>   would be a simple switch based on command-line flags.

_if_ you consider IMAP over SSL "secure", which I personally don't.
There's already support in nmh today for using the Cyrus SASL library
for different authentication mechanisms.  Using that seems to be the
right thing to do (since nearly all of the IMAP servers are going in
that direction).

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[John Reinhagen]

In message <[EMAIL PROTECTED]>, also sprach Jerry Peek:

>In nmh, multiple users can share the same mail folder.  Each user
>typically has their own private context for that shared folder stored
>in their own $HOME/Mail/context file.  So different users can see a
>different view of that folder's sequences.

True.  This argues for sequence tagging based upon a user ID.  Thanks for
mentioning this; I'd probably have missed it.

>Then there's the question of a "session": doesn't IMAP have the idea
>of "logging on" or "connecting" to an IMAP store for some period of
>time, and preserving the state of that session while the user is
>logged on?

Ugh.  This is much harder to deal with.  Superficially it seems like an easy
problem: Just handle session-related stuff in the backend.  However, there's
an essential difference in how IMAP and nmh deal with things at an atomicity
level.  A high-overhead solution is to open up an IMAP session for each
command and then close it, but that doesn't necessarily solve the problem of
simultaneous processes accessing the same IMAP folder/message.

I think IMAP does address your concerns about persistent state storage
between sessions, but I'll check.  It does not do it by allowing arbitrary
file uploads (imapfs, anyone? :), but I think tags do persist.

JCR
--
Observed by Jeff Cooper:
"Headline in the _Arizona Republic_:
247 POUNDS OF COCAINE SEIZED:  TWO HELD
 (Cheap!  Ten percent is little enough.)"

Re: [Fwd: Re: Questions about IMAP and sequences]

[John Reinhagen]

In message <[EMAIL PROTECTED]>, also sprach clemensF:

>> John Reinhagen:
>
>> Storing IMAP sequences in one's local filesystem, as Jerry Peek appears to
>> be suggesting, is another approach, but it has problems.  There are some
>> trivial annoyances -- IMAP folders can't have the same name as local
>> folders, for example.  However, there's a more fundamental bug:  The entire
>> advantage of IMAP lies in storing parts of one's email environment -- unseen
>> messages, etc. -- on a central server that one can access from anywhere.
>> Storing sequences on local filesystems violates that idea to the extent that
>> people consider sequences to be a part of their email environment.  I submit
>> that people would so consider.
>
>the error of which would be that local sequences can get out of sync when
>files are chenged on the imap-server or even on the local server.

IMAP tags follow individual messages, so yeah, the tags would follow the
messages on the IMAP server.  Or was this what you meant?

>there's no valid reverse mapping from sequences back to imap-folder-contents,
>because imap has no sequences, and if it had, it does not belong to
>nmh.  it's more like the other way around.

I wouldn't suggest implementing sequences through imap-folder-contents.

JCR
--
Observed by Jeff Cooper:
"Headline in the _Arizona Republic_:
247 POUNDS OF COCAINE SEIZED:  TWO HELD
 (Cheap!  Ten percent is little enough.)"

Re: [Fwd: Re: Questions about IMAP and sequences]

[clemensF]

> John Reinhagen:

> Storing IMAP sequences in one's local filesystem, as Jerry Peek appears to
> be suggesting, is another approach, but it has problems.  There are some
> trivial annoyances -- IMAP folders can't have the same name as local
> folders, for example.  However, there's a more fundamental bug:  The entire
> advantage of IMAP lies in storing parts of one's email environment -- unseen
> messages, etc. -- on a central server that one can access from anywhere.
> Storing sequences on local filesystems violates that idea to the extent that
> people consider sequences to be a part of their email environment.  I submit
> that people would so consider.

the error of which would be that local sequences can get out of sync when
files are chenged on the imap-server or even on the local server.
there's no valid reverse mapping from sequences back to imap-folder-contents,
because imap has no sequences, and if it had, it does not belong to
nmh.  it's more like the other way around.

clemens

Re: [Fwd: Re: Questions about IMAP and sequences]

[clemensF]

> Dan Harkless:

> Yeah, your thoughts really make it clear how much potential work is here.
> Sounds like most of the issues arise from cache handling, though.  Perhaps a
> first implementation could do everything live on the IMAP server.

whenever making copies... that's a generalisation of malloc and garbage
collection.

clemens

Re: [Fwd: Re: Questions about IMAP and sequences]

[clemensF]

> Shantonu Sen:

.
.
> 9) Subfolders!: IMAP supports subfolders. Should we? What would the
>   notation and data representation be? Any backward-compatibility
>   problems? Speaking of which, how would local caches be stored?
>   In a folder called "IMAP:[EMAIL PROTECTED]"? Yuck. Auxiliary
>   files to keep track of mappings?

i'd say files holding sequences.  they'd map to subfolders, if the
term sequence would be a tuple {name, folder, server, msg#, file-id}.

> 10) Authentication: We should leave hooks in for varying authentication
>   schemes so we don't need to make lots of changes later. Things
>   that come to mind include IMAP over SSL. Hopefully, the authentication
>   would be a simple switch based on command-line flags.

yeah, that's ok.  openssl is developed currently and might incorporate
amp and other authorization schemes in the future.

clemens

Re: [Fwd: Re: Questions about IMAP and sequences]

[clemensF]

> Ken Hornstein:

> I know of a few that use the storage format (one message per file, each
> folder a directory), but I don't think that there are any that actually
> use any code or tools from nmh.

and what for?  look at qmail maildir format:  it comes without that stupid
.sequence file.  then again, the sequences are a special function in nmh,
if "virtualisation" started with a few little scripts to abstract these
functions, they could even end there, not much work.

all the other stuff would work with maildir-format, which is well understood
in by, e.g. qmail, which is understood by fetchmail, which has an imap-
interface.

clemens

Re: [Fwd: Re: Questions about IMAP and sequences]

[clemensF]

> Anders Eriksson:

> Are you thinking of an nmh backend to an imap server or nmh ans an imap 
> client? I'd love to see the actual storage used in nmh 'virtualized', 
> so we can have the same powerful command line interface and different 
> storages (mh files, mbx, mbox, imap, ...)

please elaborate on this.

clemens

Re: [Fwd: Re: Questions about IMAP and sequences]

[Shantonu Sen]

> I had envisaged an IMAP server which would implement
> the nmh commands as an interface to the "message store", but the client side
> would be left open to "anything that supports IMAP".

Now I get it. I think having separate files/message would be a big
win on the server because now you don't have to worry about seeking
through multi-megabytes of a mail file, and constantly keeping track
of the current message, etc. If the user asks for message 330, do a 
quick stat in the correct inbox, and bam, you're ready to start writing
data to a socket. Compared to a single-file approach, the
server would either need to read through the file and find
329 start points and then start writing (dumb), or create an
index of message => offests when it first starts up. Technically the
server could store this index so you don't need to do this much
parsing every time, but when you do, it's a pain.

I think mh-style mailboxes would make that easier. I don't know if
implementing a mail server *entirely* in nmh commands owuld work/be a
good idea, but damn would it be a neat trick - write a mail server
using mail client programs...

> I future step could be to implement IMAP in exmh, which could make
> support for sequences more pertinent, but raises raises another concern;
> how would sequences be handled when talking to an IMAP server that
> wasn't our "nmh-ified" one? Wouldn't it be more correct to just stick to the
> IMAP standard?

Yes, stick to the standard. If you think there would be features talking
to an "nmh" mail server, I would suggest writing a suite of 100-line
client programs called `inc', `scan', `forw', `repl', etc., that
open a socket to a specified server, send the text of a command to run, 
have the server execute it as the user in question, and then return
the result to the user. Doing this all over IMAP seems to be
kludgy, and you'll use the most important features.

Shantonu

Re: [Fwd: Re: Questions about IMAP and sequences]

[Jerry Peek]

On 8 September 2000 at 18:02, John Reinhagen <[EMAIL PROTECTED]> wrote:
> In message <[EMAIL PROTECTED]>, also sprach Iain
> MacDonnell:
> 
> >Jerry Peek writes:
> >: I haven't thought a lot about internals here... but couldn't an IMAP
> >: implementation just store its sequences in the context file (by
> >: default, ~/Mail/context)?  This is what commands like "mark -nopublic"
> >: do.  All nmh commands can access these sequences.
...
> However, there's a more fundamental bug:  The entire
> advantage of IMAP lies in storing parts of one's email environment -- unseen
> messages, etc. -- on a central server that one can access from anywhere.
> Storing sequences on local filesystems violates that idea to the extent that
> people consider sequences to be a part of their email environment.

I don't know enough about IMAP to give a good answer about this.
I'll write some comments and hope they're useful as food for thought.

In nmh, multiple users can share the same mail folder.  Each user
typically has their own private context for that shared folder stored
in their own $HOME/Mail/context file.  So different users can see a
different view of that folder's sequences.  The same would be true for
one user who accessed a folder on an IMAP message store from multiple
hosts: the user would have different context at each host, and that's
the way it would be.  As another example, Unix filesystem permissions
might deny a user access to some of the mail folders they might want
to read -- and IMAP access permissions (no access to a remote host,
etc.) might keep someone from seeing their IMAP store.

Then there's the question of a "session": doesn't IMAP have the idea
of "logging on" or "connecting" to an IMAP store for some period of
time, and preserving the state of that session while the user is
logged on?  (In the same way that traditional email programs are either
running or they aren't?)  Of course, nmh doesn't work this way: you can
run any nmh command from anyplace (any terminal, any script, etc.) at
any time.  Then you can go away for days and come back and type "repl"
and expect to reply to the same message in the same folder you were
reading days ago.  The system can even be rebooted; the context
(current message, current folder, sequences) is preserved.  I don't
think IMAP works that way.  For instance, with IMAP/nmh, could I type
"show" in one window, then move to another window to type "repl", then
go back to the first window and type "next" while "repl" is still
running in the second window?  Would that be opening two separate IMAP
"sessions" and confuse the IMAP server between the two windows?

Maybe there's some kludge that could be tacked onto an nmh/IMAP
implementation that would store the extra nmh info in a fairly
compatible way?  For instance, does IMAP have a pigeonhole somewhere
(a folder with a name that nmh users reserve, or something) that
could store a user's context info (from $HOME/Mail/context and
/.mh_sequences) -- and that info could be copied to/from
the IMAP store with each MH command?  You'd need to do some locking
that could make nmh commands block (and even time out?), but it
might work.

It's great that people are considering this.  But I hope that whoever
implements it has experience in using raw nmh to its limits -- not
only the experience of using a limited GUI single-user single-session
email program that uses nmh as a back-end.  For nmh at its limits,
see Chapter 9 
and Chapter 13 
of the MH book.  For info about how nmh stores its context, see
Chapter 2, .
And Chapter 2 has a figure with an overview of nmh uses of filesystem
in .

I wish I knew more about the intricacies of IMAP; I've only used it in
"clueless mode" ;-) from a GUI front-end and hacked it with JavaMail.
And sorry if this is obvious; I'm not trying to "talk down" to anyone!
I'm just trying to share my (almost) 20 years of experiences with MH.
My "gut feel" is that there may not be a perfect one-to-one mapping
between nmh/exmh and IMAP.  Some things may have to bend.  I just
hope that the implementation can preserve most of the strengths and
flexibility of nmh: it's the only email system with all this power,
and it'd be sad for the de facto IMAP implementation to hobble nmh.
(I'm not saying people want to hobble nmh; these are just thoughts.)

Whew. ;-)

Jerry
-- 
Jerry Peek, [EMAIL PROTECTED], http://www.jpeek.com/

Re: [Fwd: Re: Questions about IMAP and sequences]

[Iain MacDonnell]

"Chris Garrigues" writes:
: --==_Exmh_-118578032P
: Content-Type: text/plain; charset=us-ascii
: 
: > From:  Iain MacDonnell <[EMAIL PROTECTED]>
: > Date:  Fri, 08 Sep 2000 20:39:19 +0100
: >
: > Jerry Peek writes:
: > : On 8 September 2000 at 20:05, Iain MacDonnell <[EMAIL PROTECTED]> w
: > rote:
: > : > I'm just saying that an
: > : > IMAP imeplementation could ignore sequences except for "unseen".
: > : 
: > : I haven't thought a lot about internals here... but couldn't an IMAP
: > : implementation just store its sequences in the context file (by
: > : default, ~/Mail/context)?  This is what commands like "mark -nopublic"
: > : do.  All nmh commands can access these sequences.
: > 
: > Does IMAP actually have any concept of sequences, though? I thought that was
: > left up to the client... time to get out that RFC :)
: 
: I think that's why Jerry was suggesting to put them there.  nmh *is* the 
: client in this case.

I missed the middle of this thread, due to being too busy with other things.
It sounds like my idea of IMAP support in nmh doesn't completely coincide
with everyone else's; I had envisaged an IMAP server which would implement
the nmh commands as an interface to the "message store", but the client side
would be left open to "anything that supports IMAP". That would enable me to
use, say, Netscape Messenger, when I'm away from the office, and exmh when
I'm not.

I future step could be to implement IMAP in exmh, which could make
support for sequences more pertinent, but raises raises another concern;
how would sequences be handled when talking to an IMAP server that
wasn't our "nmh-ified" one? Wouldn't it be more correct to just stick to the
IMAP standard?

Just thinking out loud *flame-guards up:)*

~Iain

Re: [Fwd: Re: Questions about IMAP and sequences]

[John Reinhagen]

In message <[EMAIL PROTECTED]>, also sprach "Dan
Harkless":

>The last time I remember IMAP support coming up was quite awhile ago, and
>the commentary (from Richard Coleman??) was that IMAP support probably
>wouldn't be forthcoming because IMAP would probably spell the eventual death
>of [n]mh.  I don't recall a lot of posts disagreeing with that view, though
>I suspect if the subject were brought up again people would disagree.

I certainly would.  My guess is that Richard Coleman thought of MH's
existing mapping between folders and directories/messages and text files to
be a central feature.  While that is a convenient way to deal with local
storage, it doesn't seem central to me.  To me, the central feature of [n]mh
is that it does away with the monolithic mail client environment in favor of
command- and script-driven environments, and IMAP support wouldn't
compromise this.  (I admit that this reverses my position as stated on
comp.mail.mh.)

>> If not, is there any objection to a Mail::NMH Perl package that provided
>> such a capability via extension?
>
>I don't quite understand the relation between nmh and your proposed Perl
>package, but I'm sure there'd be no objections to any useful extensions to
>nmh.

I'll explain in better detail.  I intend to implement various nmh entities
-- folders, messages, context (including sequences), profile -- as Perl
objects.  Any Perl-aware front end could then use and manipulate those
objects, and one could write a good mail client using such an approach.  (A
GTK-Perl nmh client would be a nice thing to have.)

Thanks to OO, I can specify a storage class for each instance of a {Folder,
Message, Sequence} object, and I can make calls to that storage class to
fetch, move, etc., the object.  Thus, local objects get treated like classic
nmh entities, while IMAP objects implement the same functionality through other
means.

The advantage of this approach is that it doesn't depend on nmh support for
IMAP.  The disadvantage is that it might discourage the implementation of
such support unless some Perl-hater was motivated to correct the
situation...but then they'd probably just implement the same objects in
Python/Java/C++/whatever.  nmh would not likely get IMAP support either way.

If there's an existing intention to put IMAP support in nmh, I'll hold off
on putting it in my Perl objects; I'd really rather handle it all through
nmh anyway.  However, if that's not likely to happen, I will design the
objects to make this possible, and will implement it as time permits.

JCR
--
Observed by Jeff Cooper:
"Headline in the _Arizona Republic_:
247 POUNDS OF COCAINE SEIZED:  TWO HELD
 (Cheap!  Ten percent is little enough.)"

Re: [Fwd: Re: Questions about IMAP and sequences]

[John Reinhagen]

In message <[EMAIL PROTECTED]>, also sprach Iain
MacDonnell:

>Jerry Peek writes:
>: On 8 September 2000 at 20:05, Iain MacDonnell <[EMAIL PROTECTED]> wrote:
>: > I'm just saying that an
>: > IMAP imeplementation could ignore sequences except for "unseen".
>: 
>: I haven't thought a lot about internals here... but couldn't an IMAP
>: implementation just store its sequences in the context file (by
>: default, ~/Mail/context)?  This is what commands like "mark -nopublic"
>: do.  All nmh commands can access these sequences.
>
>Does IMAP actually have any concept of sequences, though? I thought that was
>left up to the client... time to get out that RFC :)

IMAP does not directly support sequences, but the IMAP standard does allow
arbitrary tags and practically all IMAP implementations support them.  One
could mark a message as being part of a sequence with such a tag.  I'm not
sure how well that'd work, but it's one way.  It's certainly how any
implementation of the unseen sequence would have to work.

Storing IMAP sequences in one's local filesystem, as Jerry Peek appears to
be suggesting, is another approach, but it has problems.  There are some
trivial annoyances -- IMAP folders can't have the same name as local
folders, for example.  However, there's a more fundamental bug:  The entire
advantage of IMAP lies in storing parts of one's email environment -- unseen
messages, etc. -- on a central server that one can access from anywhere.
Storing sequences on local filesystems violates that idea to the extent that
people consider sequences to be a part of their email environment.  I submit
that people would so consider.

JCR
--
Observed by Jeff Cooper:
"Headline in the _Arizona Republic_:
247 POUNDS OF COCAINE SEIZED:  TWO HELD
 (Cheap!  Ten percent is little enough.)"

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Shantonu Sen <[EMAIL PROTECTED]> writes:
> > How do IMAP's subfolders differs from nmh's, e.g. `refile
> > +inbox/tasks'.  Or are you just saying nmh's IMAP support would include
> > IMAP's subfolders?
> 
> I'm saying we have lots of options:
> 
>  IMAP folder on Local nmh
> host mail.foo.com folder
> -   -
> work/   ~/Mail/IMAP:[EMAIL PROTECTED]/ with `work' sequence
> work/   ~/Mail/IMAP:[EMAIL PROTECTED]/work
> 
> work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/ with `work/junk' sequence
> work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/work with `junk' sequence
> work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/work/junk
> 
> So, do IMAP folders and subfolders map directly to nmh folders,
> or do we use an alternate scheme, as was earlier suggested with
> sequences?

Seems like a one-to-one mapping would be the most natural, the easiest to
implement, etc., no?

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

Shantonu Sen <[EMAIL PROTECTED]> writes:
> Here are some thoughts about IMAP support in nmh. Wherever you see "should",
> read it as "Shantonu thinks it should":

I don't know enough about IMAP to have any thoughts on some of your points,
but...

> 2) Multiple access points: Since IMAP is based on the idea of checking
>   mail from multiple locations/clients, nmh should be at *least*
>   smart enough to do the opposite, which is to check multiple
>   mail boxes at the same time from the same account. Currently,
>   this support is fairly kludgy, and half depends on compile-time
>   options, and half on command-line flags. nmh should be able
>   to check the local mail spool and IMAP without changing any
>   conf files, and the syntax should be straightforward (maybe
>   something like aliases so that "inc home" would get my IMAP
>   mail like "inc IMAP:ssen:[EMAIL PROTECTED]:INBOX")

Yeah, I suspect a lot of people are like me and have external wrapper
scripts (mine are called "check_mail" and "inca" and integrate with xbiff)
that make the multi-mailbox support nicer.  I've been wanting to enhance
nmh's native support for this.  Not sure if you were implying this or not,
but I don't think inc should always include from all defined mailboxes --
one should be able to include from a single one (or a couple) if desired.

> 3) Cache Messages: Should nmh actually download files to cache messages
>   locally? Caching obviously has performance advantages for big files
>   and files that are accessed repeatedly. However, there are
>   big wins if nmh just sits as a front-end to an IMAP connection,
>   and doesn't not store local files for IMAP accounts. We wouldn't
>   have to make many big changes - in fact, the big things would 
>   be hooks into scan and show to look over the network rather
>   than locally. Also, there would never be any synchronization
>   problems with the server, since everything is done on the server.
>   Oh, refile could also be modified to move across IMAP folders.
>   That being said, I think local caches are the more "nmh-y"
>   thing to do, and would allow users to have more control over their
>   individual emails.

If local caching is implemented, ideally it'd be disablable, so people could
optimize for slow or fast network connections.

> 4) Role of `inc': In parallel with 3) is what the role of inc would be.
>   Would it just download an index of new (or unread, depending
>   on implementation) messages? Alternatively, would it download
>   a completely new index to make sure it's completely in sync?
>   
>   Instead, it might not use the advanced IMAP features and just
>   use the server like a POP machine, and download all the new
>   messages and display a regular scan after it's done.
> 
>   As a third option, it could download an index for immediate
>   user feedback, and then fork into the background and start
>   downloading messages. If the user requests a particular email
>   via show, the daemon collector would drop it's current transfer
>   and make a new request just for that message. If we don't cache
>   messages, we don't need to worry about this.
> 
>   The forking thing is messy and contrary to nmh's style, but it's
>   what I would want as a user: Immediate feedback of new messages,
>   and then a return to the prompt so that I can decide what to
>   to next, with minimal delay.

That forking behavior would be very nice.  Presumably trying to look at
message that hasn't yet been downloaded would block until it was.

> 9) Subfolders!: IMAP supports subfolders. Should we? What would the
>   notation and data representation be? Any backward-compatibility
>   problems? Speaking of which, how would local caches be stored?
>   In a folder called "IMAP:[EMAIL PROTECTED]"? Yuck. Auxiliary
>   files to keep track of mappings?

Well, nmh already supports subfolders using subdirectory notation, so I
don't see why this should be different for IMAP support.

> 10) Authentication: We should leave hooks in for varying authentication
>   schemes so we don't need to make lots of changes later. Things
>   that come to mind include IMAP over SSL. Hopefully, the authentication
>   would be a simple switch based on command-line flags.
> 
> Are these valid challenges? I think it would definitely be neat, but
> will take a lot of effort (depending on what level of support and
> integration we achieve/try to achieve).

Yeah, your thoughts really make it clear how much potential work is here.
Sounds like most of the issues arise from cache handling, though.  Perhaps a
first implementation could do everything live on the IMAP server.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not pos

Re: [Fwd: Re: Questions about IMAP and sequences]

[Iain MacDonnell]

Jerry Peek writes:
: On 8 September 2000 at 20:05, Iain MacDonnell <[EMAIL PROTECTED]> wrote:
: > I'm just saying that an
: > IMAP imeplementation could ignore sequences except for "unseen".
: 
: I haven't thought a lot about internals here... but couldn't an IMAP
: implementation just store its sequences in the context file (by
: default, ~/Mail/context)?  This is what commands like "mark -nopublic"
: do.  All nmh commands can access these sequences.

Does IMAP actually have any concept of sequences, though? I thought that was
left up to the client... time to get out that RFC :)

~Iain

Re: [Fwd: Re: Questions about IMAP and sequences]

[Jerry Peek]

On 8 September 2000 at 20:05, Iain MacDonnell <[EMAIL PROTECTED]> wrote:
> I'm just saying that an
> IMAP imeplementation could ignore sequences except for "unseen".

I haven't thought a lot about internals here... but couldn't an IMAP
implementation just store its sequences in the context file (by
default, ~/Mail/context)?  This is what commands like "mark -nopublic"
do.  All nmh commands can access these sequences.

Jerry
-- 
Jerry Peek, [EMAIL PROTECTED], http://www.jpeek.com/

Re: [Fwd: Re: Questions about IMAP and sequences]

[Iain MacDonnell]

Scott Blachowicz writes:
: On Fri, Sep 08, 2000 at 03:35:54PM +0100, Iain MacDonnell wrote:
: >
: > Eww, adding sequences could get complex. Whilst I suppose it might be nice
: > for some, do we really need it?
: 
: I would think so...for my part, I use the "unseen" sequence a lot.

Yeah, but unseen is a special case - IMAP has the concept of unseen messages
too, so you'd want to implement that, rather than making your unseen messages
appear to be a sub-folder.

:  I also have a "picks" script:
: 
:   #!/bin/sh
:   pick -seq picked ${1+"$@"} && scan picked
: 
: that I use a lot to do things like:
: 
:   picks -subj IMAP
: 
: then if I'm not interested in them, I can 'rmm' them all at once or
: 'refile' them or whatever by sequence name.

Uh huh, me too ... but you could still do that - I'm just saying that an
IMAP imeplementation could ignore sequences except for "unseen".

~Iain

Re: [Fwd: Re: Questions about IMAP and sequences]

[Scott Blachowicz]

On Fri, Sep 08, 2000 at 03:35:54PM +0100, Iain MacDonnell wrote:
>
> Eww, adding sequences could get complex. Whilst I suppose it might be nice
> for some, do we really need it?

I would think so...for my part, I use the "unseen" sequence a lot.  I
also have a "picks" script:

#!/bin/sh
pick -seq picked ${1+"$@"} && scan picked

that I use a lot to do things like:

picks -subj IMAP

then if I'm not interested in them, I can 'rmm' them all at once or
'refile' them or whatever by sequence name.

-- 
Scott Blachowicz

Re: [Fwd: Re: Questions about IMAP and sequences]

[Iain MacDonnell]

Shantonu Sen writes:
: > How do IMAP's subfolders differs from nmh's, e.g. `refile
: > +inbox/tasks'.  Or are you just saying nmh's IMAP support would include
: > IMAP's subfolders?
: 
: I'm saying we have lots of options:
: 
:  IMAP folder on Local nmh
: host mail.foo.com folder
: -   -
: work/   ~/Mail/IMAP:[EMAIL PROTECTED]/ with `work' sequence
: work/   ~/Mail/IMAP:[EMAIL PROTECTED]/work
: 
: work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/ with `work/junk' sequence
: work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/work with `junk' sequence
: work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/work/junk
: 
: So, do IMAP folders and subfolders map directly to nmh folders,
: or do we use an alternate scheme, as was earlier suggested with
: sequences?

Eww, adding sequences could get complex. Whilst I suppose it might be nice
for some, do we really need it?

Something to bear in mind with folders is that MH folders can contain both
messages and sub-folders, where other formats would not enable this - folders
usually only contain entirely messages, or entirely sub-folders. I think
I remember Nutscrape messenger having an option for this...

~Iain

Re: [Fwd: Re: Questions about IMAP and sequences]

[Shantonu Sen]

> How do IMAP's subfolders differs from nmh's, e.g. `refile
> +inbox/tasks'.  Or are you just saying nmh's IMAP support would include
> IMAP's subfolders?

I'm saying we have lots of options:

 IMAP folder on Local nmh
host mail.foo.com folder
-   -
work/   ~/Mail/IMAP:[EMAIL PROTECTED]/ with `work' sequence
work/   ~/Mail/IMAP:[EMAIL PROTECTED]/work

work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/ with `work/junk' sequence
work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/work with `junk' sequence
work/junk   ~/Mail/IMAP:[EMAIL PROTECTED]/work/junk

So, do IMAP folders and subfolders map directly to nmh folders,
or do we use an alternate scheme, as was earlier suggested with
sequences?

Shantonu

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ralph Corderoy]

Hi,

> 9) Subfolders!: IMAP supports subfolders. Should we? What would the
> notation and data representation be? Any backward-compatibility
> problems? Speaking of which, how would local caches be stored?  In a
> folder called "IMAP:[EMAIL PROTECTED]"? Yuck. Auxiliary files to keep
> track of mappings?

How do IMAP's subfolders differs from nmh's, e.g. `refile
+inbox/tasks'.  Or are you just saying nmh's IMAP support would include
IMAP's subfolders?


Ralph.

Re: [Fwd: Re: Questions about IMAP and sequences]

[Shantonu Sen]

> I've heard all of the naysayer arguments before, but really, it wouldn't
> be _that_ hard.  Well, okay, I can see some features being lost, but
> most of the stuff would just sort of fall out once you wrote the
> glue for the backend functions.

Here are some thoughts about IMAP support in nmh. Wherever you see "should",
read it as "Shantonu thinks it should":

1) Backward compatibility: a new, theoretical version of nmh that uses
nmh should be smart enough to use old storage formats if they
already exist. If we implement a new storage backend, it should
be used only if the user starts using advanced features (like
IMAP) that require it. Upgrade and downgrade should be simple.

2) Multiple access points: Since IMAP is based on the idea of checking
mail from multiple locations/clients, nmh should be at *least*
smart enough to do the opposite, which is to check multiple
mail boxes at the same time from the same account. Currently,
this support is fairly kludgy, and half depends on compile-time
options, and half on command-line flags. nmh should be able
to check the local mail spool and IMAP without changing any
conf files, and the syntax should be straightforward (maybe
something like aliases so that "inc home" would get my IMAP
mail like "inc IMAP:ssen:[EMAIL PROTECTED]:INBOX")

3) Cache Messages: Should nmh actually download files to cache messages
locally? Caching obviously has performance advantages for big files
and files that are accessed repeatedly. However, there are
big wins if nmh just sits as a front-end to an IMAP connection,
and doesn't not store local files for IMAP accounts. We wouldn't
have to make many big changes - in fact, the big things would 
be hooks into scan and show to look over the network rather
than locally. Also, there would never be any synchronization
problems with the server, since everything is done on the server.
Oh, refile could also be modified to move across IMAP folders.
That being said, I think local caches are the more "nmh-y"
thing to do, and would allow users to have more control over their
individual emails.

4) Role of `inc': In parallel with 3) is what the role of inc would be.
Would it just download an index of new (or unread, depending
on implementation) messages? Alternatively, would it download
a completely new index to make sure it's completely in sync?

Instead, it might not use the advanced IMAP features and just
use the server like a POP machine, and download all the new
messages and display a regular scan after it's done.

As a third option, it could download an index for immediate
user feedback, and then fork into the background and start
downloading messages. If the user requests a particular email
via show, the daemon collector would drop it's current transfer
and make a new request just for that message. If we don't cache
messages, we don't need to worry about this.

The forking thing is messy and contrary to nmh's style, but it's
what I would want as a user: Immediate feedback of new messages,
and then a return to the prompt so that I can decide what to
to next, with minimal delay.

5) Message integrity: If we decide to cache, how do we decide a cached
message is the same as one on the server? Matching subject lines?
Hopefully not. Do IMAP servers have builtin methods for this?
Maybe some hidden headers? Maybe using Message IDs? MD5 (we
already have code for this for using APOP authentication)? Even
if we use something clever like Message IDs, what if the local
copy is modified? Do we consider them equivalent even in their
non-equivalence (MD5 would come in here). If we can't establish
equivalence without downloading the entire message to read
the Message ID or calculate an MD5 sum, we're in trouble, because
there goes any benefit of caching.

6) Refiling: What if a user refiles a message? Does it get immediately
updated in the IMAP server, or does it wait for the next `inc',
which is a sort of synchronization step. Dynamic refiling will
incur natural delays to bring up the connection and then
take it down again. A persistent connection to the IMAP server
would speed things up, but is not the "nmh-y" thing to do.

What if the message is refiled to another server, or just to a
local mailbox? Is it then deleted and expunged from the original
server? Again, immediately, or at the next inc?

7) Server changes: If the messages on the server changed (for example, you
delete some spam while checking your mail from the office), does
the local copy of that message get d

Re: [Fwd: Re: Questions about IMAP and sequences]

[Ken Hornstein]

>The last time I remember IMAP support coming up was quite awhile ago, and
>the commentary (from Richard Coleman??) was that IMAP support probably
>wouldn't be forthcoming because IMAP would probably spell the eventual death
>of [n]mh.  I don't recall a lot of posts disagreeing with that view, though
>I suspect if the subject were brought up again people would disagree.

I've always felt that _that_ particular view, was in a word, "crap".
The opposite is probably true; eventual lack of IMAP support might
be the death of nmh!

I've heard all of the naysayer arguments before, but really, it wouldn't
be _that_ hard.  Well, okay, I can see some features being lost, but
most of the stuff would just sort of fall out once you wrote the
glue for the backend functions.

>The only other thing I remember from that discussion was that there were
>IMAP implementations out there that used [n]mh as a back-end.

I know of a few that use the storage format (one message per file, each
folder a directory), but I don't think that there are any that actually
use any code or tools from nmh.

--Ken

Re: [Fwd: Re: Questions about IMAP and sequences]

[Dan Harkless]

John Reinhagen <[EMAIL PROTECTED]> writes:
> Is there a plan to implement IMAP connectivity in nmh?  If so, who's working
> on it and when might it be out?  

I haven't heard of anyone working on that, but unfortunately nmh developers
often have a tendency not to announce what they're working on, so it's
possible that someone is.

The last time I remember IMAP support coming up was quite awhile ago, and
the commentary (from Richard Coleman??) was that IMAP support probably
wouldn't be forthcoming because IMAP would probably spell the eventual death
of [n]mh.  I don't recall a lot of posts disagreeing with that view, though
I suspect if the subject were brought up again people would disagree.

The only other thing I remember from that discussion was that there were
IMAP implementations out there that used [n]mh as a back-end.

> If not, is there any objection to a Mail::NMH Perl package that provided
> such a capability via extension?

I don't quite understand the relation between nmh and your proposed Perl
package, but I'm sure there'd be no objections to any useful extensions to
nmh.

---
Dan Harkless   | To prevent SPAM contamination, please 
[EMAIL PROTECTED]  | do not post this private email address
SpeedGate Communications, Inc. | to the USENET or WWW.  Thank you. 

Re: [Fwd: Re: Questions about IMAP and sequences]

[Jerry Peek]

On 4 September 2000 at 11:11, John Reinhagen <[EMAIL PROTECTED]> wrote:
> MH makes a pretty strong assumption that one's file/message
> hierarchy is reflected by one's local directory/file hierarchy.
> Disregarding that, there's no problem. :)
> 
> Is there a plan to implement IMAP connectivity in nmh?  If so, who's working
> on it and when might it be out?  If not, is there any objection to a
> Mail::NMH Perl package that provided such a capability via extension?

I'm replying to this message to say that, even though John originally
sent his message to me, I'm not sure of the answer... I wish I kept up
to date on nmh sources, and the plans for nmh development, but I don't.
A lot of people have asked about IMAP and MH/nmh over the past few
years, though... and I've heard of people trying to do what John is...
does anyone have news?

Jerry
-- 
Jerry Peek, [EMAIL PROTECTED], http://www.jpeek.com/

Re: [Fwd: Re: Questions about IMAP and sequences]

[Anders Eriksson]

Are you thinking of an nmh backend to an imap server or nmh ans an imap 
client? I'd love to see the actual storage used in nmh 'virtualized', 
so we can have the same powerful command line interface and different 
storages (mh files, mbx, mbox, imap, ...)
 

/Anders


[EMAIL PROTECTED] said:
> Is there a plan to implement IMAP connectivity in nmh?  If so, who's
> working on it and when might it be out?  If not, is there any
> objection to a Mail::NMH Perl package that provided such a capability
> via extension? 




 PGP signature