Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
On 2020-07-08 18:22, Carsten Schoenert wrote: Am 08.07.20 um 23:51 schrieb Gary Dale: Do you think I want to do it this way? This is how Thunderbird makes me. I've no idea how you do this or want to do this, I was just showing how I simply drag an shove folders for years as it works for years. No matter if I do this between two active accounts or if I shove the folder to the local account. I've migrated a few POP3 account to IMAP over the time and long ago, also with some more complex folder structures, I can't remember that was that difficult in the end. I do it the way you say works. It doesn't. I have to create the first level folder structure in the IMAP account, I don't have to do this, I simply drag the folder ... ...>> I've played around with the behavior of .sdb folder together with .msf If I drag the folder, it creates the folder in the new location but doesn't copy the e-mail. Nor does it include subfolders. Dragging is just another way of creating the folders. files also long ago. Basically you can copy the .sdb folder anywhere, you also need to create an equally named empty file .msf too then, the folder will already then be visible so far I remember, once you tell TB to compress the folder TB will recreate the index data for the folder and all emails are readable again. Except that Thunderbird deletes everything it didn't create. So there are no files for it to reindex. I guess you will find a lot of websites and blogs that explain in deep how to do this if you want or need to do this at that low level side. I will not figure out this for you as this all isn't a Debian issue right now for. Perhaps it's not a Debian issue but it is a Thunderbird issue. I'm running it on Debian so if you are sure it's not Debian specific then shouldn't you pass the issue up the line? However it seems like Thunderbird has its own ideas of what is proper. I suspect that its getting it folder information from the remote server if it's not storing it locally. Either way the folders are wiped out. You again assuming and assuming, that's isn't helpful. No, the data from the email server is getting downloaded by POP3 or IMAP protocol and then stored locally. Turn on the debug mode and you will this for every single email. I don't need a debug mode to see what it does. Debug mode may show the how/why but the effects are still clear. ... The Debian wiki has some more hints about issue reporting, in case you haven't noted this already. https://wiki.debian.org/Thunderbird#Bug_Reporting_.2F_Issues There are also some hint on debugging activity, you will need to replace references to icedove with thunderbird mostly. https://wiki.debian.org/Icedove#Debugging The reference to icedove is probably just the folder used to store e-mail. The fact that it says icedove just shows when the accounts were set up in Thunderbird. Before then I was using Netscape with it's integrated mail program - when it got split up, I had to recreate the account setup but mostly kept the mail folders. I have a few newer accounts that go into .icedove but others actually use a very old Netscape file structure on a shared device. I tried moving the IMAP box to .icedove but that doesn't seem to have helped with the problems - not even noticeably speeding up the copies. You don't even have seemed tried to use this possibility to debug what Thunderbird is doing ... I was only saying that you will need to adjust the call for this kind of debugging. The only add-on I have is the one to manually sort the accounts, something that Thunderbird probably should do by itself - seems like an almost trivial operation... People always do differ on such things, and as a corporation the decision makers have to draw a line. If costumers pay the developers for integrating new features it might happen, depending on the priorities. This situation has changed for Thunderbird dramatically as Mozilla has decided that Thunderbird isn't a core part anymore. Imagine if Dolphin developers took the view that dragging a folder to a new location isn't Dolphin's job? Most dialogues that have folder views allow for simple drag & drop rearrangement. The reason I'm going through this is because Yahoo doesn't let me send mail from the POP3 account and Thunderbird stops downloading POP3 e-mail every couple of days. It seemed like time to bite the bullet and switch to IMAP but the process is far more painful than it really should be. Well as said, Yahoo is also a kind of special like Hotmail in the past or Outlook365 in newer days. It's far away to use RFC conform standards. So don't be surprised if not all is working well except you use the webUI. I could ask why doesn't Thunderbird support switching from POP3 to IMAP (and vice versa). On the face of it, it looks like simply changing some names and values - which is what you also seem to be claiming. Instead this is a multi-day effort that I am only partway through. These are all
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
Am 08.07.20 um 23:51 schrieb Gary Dale: > Do you think I want to do it this way? This is how Thunderbird makes me. I've no idea how you do this or want to do this, I was just showing how I simply drag an shove folders for years as it works for years. No matter if I do this between two active accounts or if I shove the folder to the local account. I've migrated a few POP3 account to IMAP over the time and long ago, also with some more complex folder structures, I can't remember that was that difficult in the end. > I have to create the first level folder structure in the IMAP account, I don't have to do this, I simply drag the folder ... ...>> I've played around with the behavior of .sdb folder together with .msf >> files also long ago. Basically you can copy the .sdb folder anywhere, >> you also need to create an equally named empty file .msf too then, the >> folder will already then be visible so far I remember, once you tell TB >> to compress the folder TB will recreate the index data for the folder >> and all emails are readable again. > Except that Thunderbird deletes everything it didn't create. So there > are no files for it to reindex. I guess you will find a lot of websites and blogs that explain in deep how to do this if you want or need to do this at that low level side. I will not figure out this for you as this all isn't a Debian issue right now for. > However it seems like Thunderbird has its own ideas of what is proper. I > suspect that its getting it folder information from the remote server if > it's not storing it locally. Either way the folders are wiped out. You again assuming and assuming, that's isn't helpful. No, the data from the email server is getting downloaded by POP3 or IMAP protocol and then stored locally. Turn on the debug mode and you will this for every single email. ... >> The Debian wiki has some more hints about issue reporting, in case you >> haven't noted this already. >> >> https://wiki.debian.org/Thunderbird#Bug_Reporting_.2F_Issues >> >> There are also some hint on debugging activity, you will need to replace >> references to icedove with thunderbird mostly. >> >> https://wiki.debian.org/Icedove#Debugging >> > The reference to icedove is probably just the folder used to store > e-mail. The fact that it says icedove just shows when the accounts were > set up in Thunderbird. Before then I was using Netscape with it's > integrated mail program - when it got split up, I had to recreate the > account setup but mostly kept the mail folders. I have a few newer > accounts that go into .icedove but others actually use a very old > Netscape file structure on a shared device. I tried moving the IMAP box > to .icedove but that doesn't seem to have helped with the problems - not > even noticeably speeding up the copies. You don't even have seemed tried to use this possibility to debug what Thunderbird is doing ... I was only saying that you will need to adjust the call for this kind of debugging. > The only add-on I have is the one to manually sort the accounts, > something that Thunderbird probably should do by itself - seems like an > almost trivial operation... People always do differ on such things, and as a corporation the decision makers have to draw a line. If costumers pay the developers for integrating new features it might happen, depending on the priorities. This situation has changed for Thunderbird dramatically as Mozilla has decided that Thunderbird isn't a core part anymore. > The reason I'm going through this is because Yahoo doesn't let me send > mail from the POP3 account and Thunderbird stops downloading POP3 e-mail > every couple of days. It seemed like time to bite the bullet and switch > to IMAP but the process is far more painful than it really should be. Well as said, Yahoo is also a kind of special like Hotmail in the past or Outlook365 in newer days. It's far away to use RFC conform standards. So don't be surprised if not all is working well except you use the webUI. > I could ask why doesn't Thunderbird support switching from POP3 to IMAP > (and vice versa). On the face of it, it looks like simply changing some > names and values - which is what you also seem to be claiming. Instead > this is a multi-day effort that I am only partway through. These are all question you please have to ask to upstream, these all aren't an Debian issue in my eyes. > re. testing: I don't see any reason why you can't test this between two > IMAP accounts but then Thunderbird seems to be doing lots of > unreasonable things. I've tested this and here it is working. As otherwise I would not write it into an email -- Regards Carsten
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
On 2020-07-08 15:47, Carsten Schoenert wrote: Am 08.07.20 um 20:09 schrieb Gary Dale: Have you tried this recently? Sure, otherwise I wouldn't have written it. I've done this many times over the years. Right now, I can copy a single folder (apparently) but it is empty in the IMAP inbox. I have to then copy the e-mail the POP3 folder contains. I will further note that only the first copied folder shows up even after I restart Thunderbird. No need to do single step copies here and I never have done such things this way. This is basically the change visible by git status if I drag and push a folder "Debconf Announce" from my T-Online account to my GMail account. Do you think I want to do it this way? This is how Thunderbird makes me. I have to create the first level folder structure in the IMAP account, then restart Thunderbird so I can actually see them. Thunderbird doesn't show them until it gets restarted. Then I can create the second level folders Once the folders are finally created, I can copy the e-mail into the appropriate folders but this is a long and tedious operation because it can only be done one folder at a time. $ LANG= git status --short Refresh index: 100% (622/622), done. M "ImapMail/imap.t-online.de/INBOX.sbd/Mailings.sbd/Debconf Announce.msf" M ImapMail/imap.t-online.de/INBOX.sbd/Mailings.sbd/Debian-Mails.msf M global-messages-db.sqlite M session.json M xulstore.json ?? ImapMail/imap.googlemail.com/[Google Mail].sbd/Debconf Announce.msf The file ImapMail/imap.googlemail.com/[Google Mail].sbd/Debconf Announce.msf with the mbox format is created. $ file ImapMail/imap.googlemail.com/\[Google\ Mail\].sbd/Debconf\ Announce.msf ImapMail/imap.googlemail.com/[Google Mail].sbd/Debconf Announce.msf: Mozilla Mork database, version 1.4 ... Sorry, but again this doesn't work. I noted that Thunderbird does create a .sdb folder for folders that contain other folders, but not for folders that only contain e-mail. This is the same for both mail account types but copying the folders from a POP3 account to an IMAP one results in the removal of the folders and files the next time Thunderbird starts up. I don't have any more POP3 based account for a long time so I can't readjust your specific case. But this shouldn't matter, the storage engine is the same, the mbox file format. I've played around with the behavior of .sdb folder together with .msf files also long ago. Basically you can copy the .sdb folder anywhere, you also need to create an equally named empty file .msf too then, the folder will already then be visible so far I remember, once you tell TB to compress the folder TB will recreate the index data for the folder and all emails are readable again. Except that Thunderbird deletes everything it didn't create. So there are no files for it to reindex. Thunderbird apparently only accepts folders that it creates within an IMAP mailbox. And as per my e-mail at 10:13 EDT, it doesn't even make that easy. Well, it's all done by files and folders so no deep magic here. The files and folders are the same. The created folders/files doesn't know anything about the email server over them. However it seems like Thunderbird has its own ideas of what is proper. I suspect that its getting it folder information from the remote server if it's not storing it locally. Either way the folders are wiped out. Perhaps at one point Thunderbird behaved better, but it's not doing things all that well right now. I don't think so, the engine inside Thunderbird is quite old regarding to copy or move emails between folders and or accounts. There hasn't happen invasive changes in the near past. And your report about problems is the first to this topic since I've taken over the main maintenance for Icedove/Thunderbird. If this all isn't helpful for I'm sorry I can't help you more. To my experience issues like this are in 90% grounded on bad behavior of AddOns. Similar reported issues aren't reproducible by removed or deactivated AddOns and on fresh clean profiles. As I don't know about your potential AddOns I'm also not able to help on debug your issues. But also Yahoo did change their API often in a non RFC compliant way. But the last modification on this is long ago. The Debian wiki has some more hints about issue reporting, in case you haven't noted this already. https://wiki.debian.org/Thunderbird#Bug_Reporting_.2F_Issues There are also some hint on debugging activity, you will need to replace references to icedove with thunderbird mostly. https://wiki.debian.org/Icedove#Debugging The reference to icedove is probably just the folder used to store e-mail. The fact that it says icedove just shows when the accounts were set up in Thunderbird. Before then I was using Netscape with it's integrated mail program - when it got split up, I had to recreate the account setup but mostly kept the mail folders. I have a few newer accounts that go into .icedove but others
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
Am 08.07.20 um 20:09 schrieb Gary Dale: > Have you tried this recently? Sure, otherwise I wouldn't have written it. I've done this many times over the years. > Right now, I can copy a single folder (apparently) but it is empty in > the IMAP inbox. I have to then copy the e-mail the POP3 folder > contains. I will further note that only the first copied folder shows > up even after I restart Thunderbird. No need to do single step copies here and I never have done such things this way. This is basically the change visible by git status if I drag and push a folder "Debconf Announce" from my T-Online account to my GMail account. > $ LANG= git status --short > Refresh index: 100% (622/622), done. > M "ImapMail/imap.t-online.de/INBOX.sbd/Mailings.sbd/Debconf Announce.msf" > M ImapMail/imap.t-online.de/INBOX.sbd/Mailings.sbd/Debian-Mails.msf > M global-messages-db.sqlite > M session.json > M xulstore.json > ?? ImapMail/imap.googlemail.com/[Google Mail].sbd/Debconf Announce.msf The file ImapMail/imap.googlemail.com/[Google Mail].sbd/Debconf Announce.msf with the mbox format is created. > $ file ImapMail/imap.googlemail.com/\[Google\ Mail\].sbd/Debconf\ Announce.msf > ImapMail/imap.googlemail.com/[Google Mail].sbd/Debconf Announce.msf: Mozilla > Mork database, version 1.4 ... > Sorry, but again this doesn't work. I noted that Thunderbird does create > a .sdb folder for folders that contain other folders, but not for > folders that only contain e-mail. This is the same for both mail account > types but copying the folders from a POP3 account to an IMAP one results > in the removal of the folders and files the next time Thunderbird starts up. I don't have any more POP3 based account for a long time so I can't readjust your specific case. But this shouldn't matter, the storage engine is the same, the mbox file format. I've played around with the behavior of .sdb folder together with .msf files also long ago. Basically you can copy the .sdb folder anywhere, you also need to create an equally named empty file .msf too then, the folder will already then be visible so far I remember, once you tell TB to compress the folder TB will recreate the index data for the folder and all emails are readable again. > Thunderbird apparently only accepts folders that it creates within an > IMAP mailbox. And as per my e-mail at 10:13 EDT, it doesn't even make > that easy. Well, it's all done by files and folders so no deep magic here. The files and folders are the same. The created folders/files doesn't know anything about the email server over them. > Perhaps at one point Thunderbird behaved better, but it's not doing > things all that well right now. I don't think so, the engine inside Thunderbird is quite old regarding to copy or move emails between folders and or accounts. There hasn't happen invasive changes in the near past. And your report about problems is the first to this topic since I've taken over the main maintenance for Icedove/Thunderbird. If this all isn't helpful for I'm sorry I can't help you more. To my experience issues like this are in 90% grounded on bad behavior of AddOns. Similar reported issues aren't reproducible by removed or deactivated AddOns and on fresh clean profiles. As I don't know about your potential AddOns I'm also not able to help on debug your issues. But also Yahoo did change their API often in a non RFC compliant way. But the last modification on this is long ago. The Debian wiki has some more hints about issue reporting, in case you haven't noted this already. https://wiki.debian.org/Thunderbird#Bug_Reporting_.2F_Issues There are also some hint on debugging activity, you will need to replace references to icedove with thunderbird mostly. https://wiki.debian.org/Icedove#Debugging -- Regards Carsten
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
On 2020-07-08 13:04, Carsten Schoenert wrote: Hello Gary, Am 08.07.20 um 16:22 schrieb Gary Dale: * What exactly did you do (or not do) that was effective (or ineffective)? I quickly discovered that Thunderbird only lets you copy e-mail, not folders of e-mail. WHile there used to be a plug-in to handel moving folders, it apparently doesn't work with current versions of Thunderbird. I can simply drag a folder and push it to the desired new account or subfolder. At least a single folder is than copied to the target folder/account. Long long ago were I moved away from POP3 to IMAP I moved all my local stored folders and emails within the account to the local structure. And afterwards I used the same technic to move my emails back to the newly created IMAP setup. Was working without any problems. Have you tried this recently? Right now, I can copy a single folder (apparently) but it is empty in the IMAP inbox. I have to then copy the e-mail the POP3 folder contains. And the file structures of POP3 inboxes appear to be different from the file structurs of IMAP inboxes so that copying the POP3 files outside of Thunderbird merely results in them being erased when Thunderbird restarts. No, Pop3 vs IMAP is a server related detail. Thunderbird is storing emails locally by the mbox format as default. In recent versions you can choose for new accounts to use maildir as storage back end. But the suggestion is still to use the mbox format. You can simply copy these folders and the index file manually to the target folder or system. But this is completely independent to the email server you like to use. Sorry, but again this doesn't work. I noted that Thunderbird does create a .sdb folder for folders that contain other folders, but not for folders that only contain e-mail. This is the same for both mail account types but copying the folders from a POP3 account to an IMAP one results in the removal of the folders and files the next time Thunderbird starts up. Thunderbird apparently only accepts folders that it creates within an IMAP mailbox. And as per my e-mail at 10:13 EDT, it doesn't even make that easy. Perhaps at one point Thunderbird behaved better, but it's not doing things all that well right now.
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
On 2020-07-08 13:04, Carsten Schoenert wrote: Hello Gary, Am 08.07.20 um 16:22 schrieb Gary Dale: * What exactly did you do (or not do) that was effective (or ineffective)? I quickly discovered that Thunderbird only lets you copy e-mail, not folders of e-mail. WHile there used to be a plug-in to handel moving folders, it apparently doesn't work with current versions of Thunderbird. I can simply drag a folder and push it to the desired new account or subfolder. At least a single folder is than copied to the target folder/account. Long long ago were I moved away from POP3 to IMAP I moved all my local stored folders and emails within the account to the local structure. And afterwards I used the same technic to move my emails back to the newly created IMAP setup. Was working without any problems. Have you tried this recently? Right now, I can copy a single folder (apparently) but it is empty in the IMAP inbox. I have to then copy the e-mail the POP3 folder contains. I will further note that only the first copied folder shows up even after I restart Thunderbird. And the file structures of POP3 inboxes appear to be different from the file structurs of IMAP inboxes so that copying the POP3 files outside of Thunderbird merely results in them being erased when Thunderbird restarts. No, Pop3 vs IMAP is a server related detail. Thunderbird is storing emails locally by the mbox format as default. In recent versions you can choose for new accounts to use maildir as storage back end. But the suggestion is still to use the mbox format. You can simply copy these folders and the index file manually to the target folder or system. But this is completely independent to the email server you like to use. Sorry, but again this doesn't work. I noted that Thunderbird does create a .sdb folder for folders that contain other folders, but not for folders that only contain e-mail. This is the same for both mail account types but copying the folders from a POP3 account to an IMAP one results in the removal of the folders and files the next time Thunderbird starts up. Thunderbird apparently only accepts folders that it creates within an IMAP mailbox. And as per my e-mail at 10:13 EDT, it doesn't even make that easy. Perhaps at one point Thunderbird behaved better, but it's not doing things all that well right now.
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
Hello Gary, Am 08.07.20 um 16:22 schrieb Gary Dale: * What exactly did you do (or not do) that was effective (or ineffective)? I quickly discovered that Thunderbird only lets you copy e-mail, not folders of e-mail. WHile there used to be a plug-in to handel moving folders, it apparently doesn't work with current versions of Thunderbird. I can simply drag a folder and push it to the desired new account or subfolder. At least a single folder is than copied to the target folder/account. Long long ago were I moved away from POP3 to IMAP I moved all my local stored folders and emails within the account to the local structure. And afterwards I used the same technic to move my emails back to the newly created IMAP setup. Was working without any problems. And the file structures of POP3 inboxes appear to be different from the file structurs of IMAP inboxes so that copying the POP3 files outside of Thunderbird merely results in them being erased when Thunderbird restarts. No, Pop3 vs IMAP is a server related detail. Thunderbird is storing emails locally by the mbox format as default. In recent versions you can choose for new accounts to use maildir as storage back end. But the suggestion is still to use the mbox format. You can simply copy these folders and the index file manually to the target folder or system. But this is completely independent to the email server you like to use. -- Regards Carsten
Bug#964547: thunderbird: I want to move or copy folder structures between mailboxes
Package: thunderbird
Version: 1:68.10.0-1
Severity: wishlist
Dear Maintainer,
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
I converting a very old mailbox from POP3 to IMAP. It has a lot of folders and
subfolders
* What exactly did you do (or not do) that was effective (or ineffective)?
I quickly discovered that Thunderbird only lets you copy e-mail, not folders of
e-mail. WHile there used to be a
plug-in to handel moving folders, it apparently doesn't work with current
versions of Thunderbird. And the file structures
of POP3 inboxes appear to be different from the file structurs of IMAP inboxes
so that copying the POP3 files outside of
Thunderbird merely results in them being erased when Thunderbird restarts.
* What was the outcome of this action?
It seems to me that copying a folder isn't that much more complicated that
copying e-mails. You simply need to detect
if the folder exists in the destination, and if it doesn't, create it then copy
all the e-mail from the source folder to
the destination. This should, of course, be recursive because folders can
contain other folders.
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
-- System Information:
Debian Release: bullseye/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.7.0-1-amd64 (SMP w/16 CPU cores)
Locale: LANG=iu_CA.UTF-8, LC_CTYPE=C.UTF-8 (charmap=locale: Cannot set
LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
UTF-8), LANGUAGE=en_CA:en (charmap=locale: Cannot set LC_MESSAGES to default
locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages thunderbird depends on:
ii debianutils 4.9.1
ii fontconfig2.13.1-4.2
ii libatk1.0-0 2.36.0-2
ii libc6 2.30-8
ii libcairo-gobject2 1.16.0-4
ii libcairo2 1.16.0-4
ii libdbus-1-3 1.12.20-1
ii libdbus-glib-1-2 0.110-5
ii libevent-2.1-72.1.11-stable-1
ii libffi7 3.3-4
ii libfontconfig12.13.1-4.2
ii libfreetype6 2.10.1-2
ii libgcc-s1 10.1.0-4
ii libgdk-pixbuf2.0-02.40.0+dfsg-5
ii libglib2.0-0 2.64.3-2
ii libgtk-3-03.24.20-1
ii libgtk2.0-0 2.24.32-4
ii libicu67 67.1-2
ii libjsoncpp1 1.7.4-3.1
ii libnspr4 2:4.25-1
ii libnss3 2:3.53.1-1
ii libpango-1.0-01.44.7-4
ii libsqlite3-0 3.32.3-1
ii libstartup-notification0 0.12-6
ii libstdc++610.1.0-4
ii libvpx6 1.8.2-1
ii libx11-6 2:1.6.9-2+b1
ii libx11-xcb1 2:1.6.9-2+b1
ii libxcb-shm0 1.14-2
ii libxcb1 1.14-2
ii libxext6 2:1.3.3-1+b2
ii libxrender1 1:0.9.10-1
ii libxt61:1.1.5-1+b3
ii psmisc23.3-1
ii x11-utils 7.7+5
ii zlib1g1:1.2.11.dfsg-2
Versions of packages thunderbird recommends:
ii hunspell-en-us [hunspell-dictionary] 1:2019.10.06-1
pn lightning
Versions of packages thunderbird suggests:
ii apparmor 2.13.4-3
ii fonts-lyx 2.3.5.2-1
ii libgssapi-krb5-2 1.17-10
-- debconf information excluded
-- debsums errors found:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "en_CA:en",
LC_ALL = (unset),
LC_TIME = "en_CA.UTF-8",
LC_CTYPE = "C.UTF-8",
LANG = "iu_CA.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
