Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
Oh, I meant to add that compacting is typically useful when record (email?) storage is in something like an mbox file -- it saves the need to rewrite the file each time a single file is deleted (for example). On the other hand, with storage in something like mdirr files (right name -- one email / record per file) it is not really relevant (unless those records / emails are indexed by some other indexing system that stores pointers to all of those records). In (older versions?) of kmail, with mbox storage, compactiing is necessary to reclaim unused space after deletions, and there is an option in the context menu for (most) kmail folders to initiate compaction. On Monday, May 01, 2023 08:45:15 AM rhkra...@gmail.com wrote: > On Sunday, April 30, 2023 06:24:14 PM Default User wrote: > > > What is 'compacting', what is it meant to do? > > The definition of compacting as I "grew up" with it (not sure where I first > encountered it is the idea that in some applications, the act of "deleting" > something doesn't actually delete it from the file, instead it is marked > for deletion (and no longer visible to the user). > > Compacting actually deletes that thing, and shrinks the file to avoid any > wasted (empty") space previously used by those deleted things. > > I suspect I first encountered that back in the early days of DOS based > databases, even prior to dBase II. -- rhk | No entity has permission to use this email to train an AI.
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sunday, April 30, 2023 06:24:14 PM Default User wrote: > > What is 'compacting', what is it meant to do? The definition of compacting as I "grew up" with it (not sure where I first encountered it is the idea that in some applications, the act of "deleting" something doesn't actually delete it from the file, instead it is marked for deletion (and no longer visible to the user). Compacting actually deletes that thing, and shrinks the file to avoid any wasted (empty") space previously used by those deleted things. I suspect I first encountered that back in the early days of DOS based databases, even prior to dBase II. -- rhk (sig revised 20230312 -- modified first paragraph, some other irrelevant wordsmithing) | No entity has permission to use this email to train an AI. If you reply: snip, snip, and snip again; leave attributions; avoid HTML; avoid top posting; and keep it "on list". (Oxford comma (and semi-colon) included at no charge.) If you revise the topic, change the Subject: line. If you change the topic, start a new thread. Writing is often meant for others to read and understand (legal documents excepted?) -- make it easier for your reader by various means, including liberal use of whitespace (short paragraphs, separated by whitespace / blank lines) and minimal use of (obscure?) jargon, abbreviations, acronyms, and references. If someone has already responded to a question, decide whether any response you add will be helpful or not ... A picture is worth a thousand words. A video (or "audio"): not so much -- divide by 10 for each minute of video (or audio) or create a transcript and edit it to 10% of the original. A speaker who uses ahhs, ums, or such may have a real physical or mental disability, or may be showing disrespect for his listeners by not properly preparing in advance and thinking before speaking. (That speaker might have been "trained" to do this by being interrupted often if he pauses.) (Remember Cicero who did not have enough time to write a short missive.) A radio (or TV) station which broadcasts speakers with high pitched voices (or very low pitched / gravelly voices) (which older people might not be able to hear properly) disrespects its listeners. Likewise if it broadcasts extraneous or disturbing sounds (like gunfire or crying), or broadcasts speakers using their native language (with or without an overdubbed translation). A person who writes a sig this long probably has issues and disrespects (and offends) a large number of readers. ;-) '
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Mon, May 01, 2023 at 07:38:45AM +0100, Tixy wrote: > On Sun, 2023-04-30 at 18:24 -0400, Default User wrote: > > It does occur to me that Evolution may use the maildir format rather > > than the mbox format [...] > I thought we were talking about IMAP protocol access to Google? In > which case Evolution isn't storing email in any format [1], it's > accessing it on a remote server [...] Oh, goody. I only brought mbox up as an attempt to explain how such a kind of unser interface (mark for deletion, then bulk expunge/ compact/whatyoucallit) might have developed. After that, users might expect that kind of interface regardless of the actual storage backend, so many mailers offer it. Cheers -- t signature.asc Description: PGP signature
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sun, 2023-04-30 at 18:24 -0400, Default User wrote: > It does occur to me that Evolution may use the maildir format rather > than the mbox format. I just ASSumed that it used mbox, since in Menu > > File, there is an option to save messages in mbox format. If > Evolution uses the maildir format, compacting apparently does not > apply, which would seem to explain it. I have not yet determined > whether Evolution uses mbox or maildir. I thought we were talking about IMAP protocol access to Google? In which case Evolution isn't storing email in any format [1], it's accessing it on a remote server. Evolution does have an 'On This Computer' email account, don't know what format it uses for that. In Evolution you can also create email accounts where email is stored locally in maildir or mbox format. I used maildir some years ago to have an email archive of some mailing lists. [1] Well, Evolution does cache email in ~/.cache/evolution/mail which seems to involve SQLite database and a file for each email. If you want to free that space you can just 'rm -rf ~/.cache/evolution' but I guess if you have 'download for offline use' configured or actually look at the remote email folders again it will re-download them. -- Tixy
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On 01/05/2023 05:36, Stefan Monnier wrote: According to this: https://support.mozilla.org/en-US/kb/compacting-folders This basically explains that compacting is an operation that should fundamentally be transparent to the user, and that Thunderbird makes the user aware of it for ... no good reason. This support page discusses behavior for local folders. This thread is dedicated to specific of Gmail and Evolution interaction in respect to IMAP protocols. I mentioned compacting of folders in Thnderbird because earlier I have seen that performing "Compact" in Thunderbird expunges messages earlier marked for deletion. Strike through in message list may be a sign that the message is marked for deletion. I expect that the following applies to message state on server, not to local folders: https://datatracker.ietf.org/doc/html/rfc3501#section-2.3.2 RFC 3501 IMAPv4 2.3.2. Flags Message Attribute \Deleted Message is "deleted" for removal by later EXPUNGE It is independent of maildir or mbox storage format of local folders or local offline cache of remote folder. Gmail may use internally some proprietary format that is neither mbox nor maildir, e.g. records in some database. I like that Thunderbird exposes the "compact" option to UI because I earlier used it to forced removal of large messages from server account with rather strict quota. Several times I used "undelete" to recover messages I deleted by mistake, so I like that messages just marked for deletion by default as well. The following is related neither to Evolution (at least directly) nor to Thunderbird, but still might be relevant: https://support.microsoft.com/en-us/office/restore-deleted-messages-when-using-an-imap-account-4559a297-4d46-47e7-bfbf-71287b1935ed If you use Google Gmail as an IMAP account, messages marked for deletion are permanently deleted from their original folder but may still appear there in strikethrough text. These messages are still recoverable because a copy is kept in the Gmail Trash folder. On 01/05/2023 05:24, Default User wrote: compacting does not: - delete messages - remove messages These statements may be confusing in the context of this thread. It is true that the action does not deletes messages if the user earlier has not requested removal. However in some cases it possible to undelete message before compacting, but not after. So it really deletes messages, perhaps hidden from UI (or displayed as strike through and still available).
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sun, 2023-04-30 at 20:47 +0200, to...@tuxteam.de wrote: > On Sun, Apr 30, 2023 at 06:26:53PM +0100, Tixy wrote: > > On Sun, 2023-04-30 at 11:19 -0400, Default User wrote: > > > (BTW, if anyone does have information about compacting folders in > > > Evolution, I would love to hear about it!) > > > > What is 'compacting', what is it meant to do? [...] > > > If it just means really deleting email, then you can select the > > menu > > option File > Empty Wastebasket which does that for all accounts. > > Or > > for just a single account, right click on it's Wastebasket. There > > is > > also Folder > Expunge which I believe does it for a single folder. > > This is probably the intended meaning. For strange historical reasons > (mbox), reclaiming the space of a deleted mail used to be a non > trivial > operation, so the user interface separated both: marking single mails > for deletion and actually doing it (which went by different names, > like "expunge", possibly "compact" -- the latter reminiscent of > crushing > the holes left in a linear mbox). > > That user interface proved useful for other reasons (time for > remorse), > so it stuck, probably. > > So yes, wastebasket emptying is probably it. > > Cheers Hi, Tomas. Okay, it looks like Evolution does indeed use the MAILDIR format. See: https://help.gnome.org/users/evolution/stable/import-supported-file-formats.html.en which says, "The format used by Evolution (for local folders since version 3.0). There is no need to import Maildir files as you can configure a Maildir account in Evolution and point to the folder where the Maildir files are stored." And according to: https://support.mozilla.org/en-US/kb/compacting-folders MBOX is the default format, where all of a folder's messages are stored in a single file on disk. This is where the compact process is useful, and the purpose of this article is to explain how and why. Maildir is a newer storage format, where every message of a folder is a separate file. Maildir does not need compact, and so this article is not applicable to Maildir folders. So, apparently Evolution does not need "compacting"! BTW, interesting that Thunderbird seems to still use the mbox format by default, although you can apparently choose to store messages in either mbox or maildir format. Which is why they explain and offer the choice to users.
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
> According to this: > https://support.mozilla.org/en-US/kb/compacting-folders This basically explains that compacting is an operation that should fundamentally be transparent to the user, and that Thunderbird makes the user aware of it for ... no good reason. Thunderbird will compact things for you as needed even if you don't explicitly ask for it, so you don't need to worry about it. Evolution doesn't bother the user with this, which is the better option. Stefan
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sun, 2023-04-30 at 18:26 +0100, Tixy wrote: > On Sun, 2023-04-30 at 11:19 -0400, Default User wrote: > > (BTW, if anyone does have information about compacting folders in > > Evolution, I would love to hear about it!) > > What is 'compacting', what is it meant to do? Sounds like > 'compressing' > to me, don't know if protocols like IMAP has a way to ask the server > to > compress it's folders. (I would have though how emails are actually > stored would be an internal implementation detail of the server). > > If it just means really deleting email, then you can select the menu > option File > Empty Wastebasket which does that for all accounts. Or > for just a single account, right click on it's Wastebasket. There is > also Folder > Expunge which I believe does it for a single folder. > > You can also configure Evolution to empty the wastebasket after a > period of time, or as I have it configured, whenever you close > Evolution. (I've always hated the idea of Wastebaskets, if I delete > something I want it deleted, not just hidden and taking up space). > Hi Tixy. I don't know technically what "compacting" is supposed to mean, but just as a total guess, I wonder if it is something like the 'vacuuming" operation for databases. According to this: https://support.mozilla.org/en-US/kb/compacting-folders compacting does not: - delete messages - remove messages - compress folders Note that this just says that compacting improves performance. But I believe I have read elsewhere that not compacting regularly can have potentially dire consequences, such as corrupted email storage files and lost email messages. Also consider this: https://thunderbirdtweaks.blogspot.com/2011/07/compacting-what-is-it-and-why-must-i-do.html So "expunging" emails may not be the same thing as compacting. FWIW, in Evolution I had multiple email accounts with more than 1,000 messages each, including one which had over 24,000 messages! Spending many days, deleting sometimes several hundred messages at a time, I have now reduced that to less than 6,000. No lost messages or file corruption yet . . . It does occur to me that Evolution may use the maildir format rather than the mbox format. I just ASSumed that it used mbox, since in Menu > File, there is an option to save messages in mbox format. If Evolution uses the maildir format, compacting apparently does not apply, which would seem to explain it. I have not yet determined whether Evolution uses mbox or maildir. BTW, I also never did really like the wastebasket (recycle bin) concept, in any application program. After all, delete should mean delete, right? But I must confess that the wastebasket idea has rescued me more than once from a deletion by mistake, or when I have "changed my mind"!
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sun, Apr 30, 2023 at 06:26:53PM +0100, Tixy wrote: > On Sun, 2023-04-30 at 11:19 -0400, Default User wrote: > > (BTW, if anyone does have information about compacting folders in > > Evolution, I would love to hear about it!) > > What is 'compacting', what is it meant to do? [...] > If it just means really deleting email, then you can select the menu > option File > Empty Wastebasket which does that for all accounts. Or > for just a single account, right click on it's Wastebasket. There is > also Folder > Expunge which I believe does it for a single folder. This is probably the intended meaning. For strange historical reasons (mbox), reclaiming the space of a deleted mail used to be a non trivial operation, so the user interface separated both: marking single mails for deletion and actually doing it (which went by different names, like "expunge", possibly "compact" -- the latter reminiscent of crushing the holes left in a linear mbox). That user interface proved useful for other reasons (time for remorse), so it stuck, probably. So yes, wastebasket emptying is probably it. Cheers -- t signature.asc Description: PGP signature
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sun, 2023-04-30 at 11:19 -0400, Default User wrote: > (BTW, if anyone does have information about compacting folders in > Evolution, I would love to hear about it!) What is 'compacting', what is it meant to do? Sounds like 'compressing' to me, don't know if protocols like IMAP has a way to ask the server to compress it's folders. (I would have though how emails are actually stored would be an internal implementation detail of the server). If it just means really deleting email, then you can select the menu option File > Empty Wastebasket which does that for all accounts. Or for just a single account, right click on it's Wastebasket. There is also Folder > Expunge which I believe does it for a single folder. You can also configure Evolution to empty the wastebasket after a period of time, or as I have it configured, whenever you close Evolution. (I've always hated the idea of Wastebaskets, if I delete something I want it deleted, not just hidden and taking up space). -- Tixy
Re: Evolution email (problem?) (IMAP, Gmail, email compacting)
On Sat, 2023-04-29 at 00:01 +0700, Max Nikulin wrote: > For those who missed start of the thread: it is dedicated to IMAP > access > to Gmail and Evolution behavior. > > On 27/04/2023 23:15, Default User wrote: > > It stays in both folders, with > > only the one in "All Mail" having a line through it, showing that > > is > > marked for deletion, but is not deleted, and is not even moved to > > "Trash", which remains empty. > > What will happen if you try to "compact" the "All Mail" folder? I am > unsure concerning precise name of such action in evolution. In > thunderbird delete vs. mark for deletion setting is account-wide and > can > not be chosen per folder. However even when "delete" is chosen, > messages > may be retained in folders till they are automatically compacted or > this > action is forced by user. > > > Moving the message from "Inbox" to another folder in the same email > > account, for example "Spam", DOES remove the message from "All > > Mail". > > That does not make sense to me! > > It is better to experiment with a custom label. Spam or Trash are > rather > special cases (perhaps on both sides: client and server). > > > I am reluctant to test moving a message from one email account to > > another one, as I do not do not want to have an another unexpected > > problem occur, and I do want to keep the email accounts strictly > > isolated from each other. > > Almost certainly messages may be deleted from web UI. Anyway a couple > of > test messages should not cause real mess. > Hi, Max. I can not seem to find any information about compacting folders in Evolution. Which is strange. In Thunderbird, you can manually compact folders any time, and it will compact automatically by default, with adjustable parameters. Perhaps Evolution considers "expunging" (actual deletion) of messages in the Trash folder, or otherwise marked for deletion (except in the All Mail folder), as "compacting". I sure hope that Evolution is actually compacting folders somehow, and just not bothering to explain when or how. Anyway, I think I have spent enough time on the Evolution All Mail folder issue. For now, I will just add one or more "Saved" folders, and pretty much just ignore the All Mail folder, unless for some reason I need to move messages from there to other folders. So, thanks to all who weighed in on the issue. (BTW, if anyone does have information about compacting folders in Evolution, I would love to hear about it!)
Re: Evolution email (problem?) (IMAP, Gmail)
For those who missed start of the thread: it is dedicated to IMAP access to Gmail and Evolution behavior. On 27/04/2023 23:15, Default User wrote: It stays in both folders, with only the one in "All Mail" having a line through it, showing that is marked for deletion, but is not deleted, and is not even moved to "Trash", which remains empty. What will happen if you try to "compact" the "All Mail" folder? I am unsure concerning precise name of such action in evolution. In thunderbird delete vs. mark for deletion setting is account-wide and can not be chosen per folder. However even when "delete" is chosen, messages may be retained in folders till they are automatically compacted or this action is forced by user. Moving the message from "Inbox" to another folder in the same email account, for example "Spam", DOES remove the message from "All Mail". That does not make sense to me! It is better to experiment with a custom label. Spam or Trash are rather special cases (perhaps on both sides: client and server). I am reluctant to test moving a message from one email account to another one, as I do not do not want to have an another unexpected problem occur, and I do want to keep the email accounts strictly isolated from each other. Almost certainly messages may be deleted from web UI. Anyway a couple of test messages should not cause real mess.
