Re: status of test code
Thanks for the update, we'll check it out and see if there is something to do. Aki > On 10/01/2021 23:34 st...@keptprivate.com wrote: > > > Update: > > I was finally able to build 2.3.13 on centos 8 (8.3). I managed this by going > back to the qmailtoaster spec file, removing a few things no longer relevant > (vpopmail) and disabling all the tests. Most tests were still running but > there were also many that did not. The good news is the resulting executable > in the RPM no longer complains about not being able to find libdovecot.so > (http://libdovecot.so). i've done some basic testing on imap and everything > appears to be working. > > Someone who maintains the tests and rpm build should really have a look. > Doing a build on a fresh centos 8 machine, as I was able to with the > qmailtoaster source rpm, should really be much simpler. My impression is the > current spec file may have edits that were eroneously committed? > > Steve > > Sent from my T-Mobile 4G LTE device > > -- Original message-- > From:st...@keptprivate.com > Date:Sat, Jan 9, 2021 3:42 PM > To:dovecot@dovecot.org; > Cc: > Subject:status of test code > > Hi, > > I'm continuing to try to build 2.3.13 with a source RPM. > > At this point I've taken the source zip file and I'm working with the > previously working qmailtoaster SPEC file and RPM build process. > > The toaster SPEC file runs the built-in dovecot tests after build... 2.3.11 > would make it through > all the tests with a few minor exceptions. > > 2.3.13 seems no longer able to run the test is lib-ssl-iostream or lib-lua > (and perhaps others, but that's as far as I've gotten). > > I can selectively disable the tests to make progress, but it raises the > question of what the plans are for the built-in tests. > > Also, I continue to not be able to find where all the testing is turned > on/off at once? I'm sure it will be obvious when someone tells me but > please tell me, because I'm pulling my hair out. > > Steve > >
Re: migration with doveadm backup to new cluster running dovecot 2.2.36 and replicator
we were able to narrow down the cause of the problem. After the initial dsync migration process the mailbox GUIDs are the same for each mailbox-name across all users. Is this intended behaviour of dsync? If not, how can this be changed? After the first replication process the Inboxes of ~20% of users get different mailbox GUIDs. During the next incremental dsync process the mailbox GUIDs of these ~20% get overridden by the inital one. Then the incremental replication duplicates those Inboxes where the mailbox GUIDs don't match. Any ideas? thanks Thomas On 07.01.2021 16:41, Thomas Winterstein wrote: dsync is intended to be used to change mailbox format, so it should work just fine. that's exactly what we thought and why we use dsync to migrate like described here https://wiki2.dovecot.org/Migration/Dsync Our replication is configured according to https://wiki.dovecot.org/Replication Both processes run separately in time. Still on some accounts mails of Inbox or another folder get duplicated. We're currently trying to debug this. what are we missing? thanks Thomas On 07.01.2021 10:21, Aki Tuomi wrote: dsync is intended to be used to change mailbox format, so it should work just fine. Aki On 07/01/2021 11:17 Andrea Gabellini wrote: Hello, I had a similar problem some time ago, and the problem was the mailbox format change. Please try to migrate with the same format. Andrea Il 05/01/21 15:02, Thomas Winterstein ha scritto: No one? If there are limitations in regards to how dsync in migration and replication can operate together these should be stated clearly in the documentation. On 23.12.2020 20:33, Thomas Winterstein wrote: Hello everyone, we are working on migrating from dovecot 2.0.9 (maildir) to 2.2.36 (mdbox). The new cluster has two backend mail servers which replicate through doveadm replicator. To move the data initially we use doveadm backup (imapc). arb Our migration command doveadm -o mail_fsync=never backup -R -u $user imapc: To test the replication of new and purge of old mails with live data changes we ran imapc on a daily basis but encountered the problem that some mailboxes multiplied in size. We then made sure that imapc and replication don't run at the same time but after the first incremental imapc process, we still had the same problems. The doveadm-backup man-page states that it's possible to run it multiple times during migration. But is it also possible to have the replicator running in between? From our understanding the doveadm backup should just work as an imap connection between the servers, synchronizing all changes made on the source to the destination. Or does the conversion from maildir to mdbox format in our case produce the problems? If you're not supposed to run the replicator before having fully migrated, how can we shorten the downtime? rsync? And how can we be sure that similar problems don't occur after the migration if we can't test all mechanisms together with live data? thanks -- __ Daddy, why doesn't this magnet pick up this floppy ? __ TIM San Marino S.p.A. Andrea Gabellini Engineering R TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a priv...@telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge. -- Thomas Winterstein http://www.rz.uni-augsburg.de/ Universität Augsburg, Rechenzentrum . Tel. (0821) 598-2068 86135 Augsburg .. Fax. (0821) 598-2028
Re: status of test code
Update:I was finally able to build 2.3.13 on centos 8 (8.3). I managed this by going back to the qmailtoaster spec file, removing a few things no longer relevant (vpopmail) and disabling all the tests. Most tests were still running but there were also many that did not. The good news is the resulting executable in the RPM no longer complains about not being able to find libdovecot.so. i've done some basic testing on imap and everything appears to be working.Someone who maintains the tests and rpm build should really have a look. Doing a build on a fresh centos 8 machine, as I was able to with the qmailtoaster source rpm, should really be much simpler. My impression is the current spec file may have edits that were eroneously committed? SteveSent from my T-Mobile 4G LTE device-- Original message--From: steve@keptprivate.comDate: Sat, Jan 9, 2021 3:42 PMTo: dovecot@dovecot.org;Cc: Subject:status of test codeHi, I'm continuing to try to build 2.3.13 with a source RPM. At this point I've taken the source zip file and I'm working with the previously working qmailtoaster SPEC file and RPM build process. The toaster SPEC file runs the built-in dovecot tests after build... 2.3.11 would make it through all the tests with a few minor exceptions. 2.3.13 seems no longer able to run the test is lib-ssl-iostream or lib-lua (and perhaps others, but that's as far as I've gotten). I can selectively disable the tests to make progress, but it raises the question of what the plans are for the built-in tests. Also, I continue to not be able to find where all the testing is turned on/off at once? I'm sure it will be obvious when someone tells me but please tell me, because I'm pulling my hair out. Steve
Re: lzma mdboxes with unexpected EOF
Hi, > > But I'm wondering why that happened at all. I stumpled upon [1] but > there was no further discussion about it. > it seems it was caused by a bug mentioned in 2.3.13's changelog: - lib-compression: Using xz/lzma compression in v2.3.11 could have written truncated output in some situations. This would result in "Broken pipe" read errors when trying to read it back. So, sorry for the noise :) Best Regards Bjoern
auth error log format change
Hi, we recently upgraded to 2.3.13 from 2.3.11 and the log format for auth errors seems to have change. This change may have an adverse effect on fail2ban rules. Is there a way to revert to old format ? Thank you ! Old log example: Jan 10 00:32:37 mail dovecot: auth-worker(50806): sql(name@domain,112.18.92.0.104,): Password mismatch Jan 10 03:20:02 mail dovecot: auth-worker(26057): sql(name@domain,112.18.92.0.104,): Password mismatch New log example: Jan 10 00:32:37 mail dovecot: auth-worker(50806): conn unix:auth-worker (pid=22503,uid=0): auth-worker<585236>: sql(name@domain,112.18.92.0.104,): Password mismatch Jan 10 03:20:02 mail dovecot: auth-worker(26057): conn unix:auth-worker (pid=25928,uid=0): auth-worker<4234337>: sql(name@domain,112.18.92.0.104,): Password mismatch -- Best regards, Adrian Minta
lzma mdboxes with unexpected EOF
Hi, I'm running dovecot 2.3.13 (until 05.01. it was 2.3.11.3) on Arch Linux and used lzma compressed mdboxes. Yesterday a user called me that she is missing all mails from the beginning of the month until now. I took a look into the logs and saw several "lzma.read [...] unexpected EOF at <>". As I had no clue why the mdboxes are broken, I tried to convert all 3 mailboxes to Maildir using "doveadm sync" to prevent further issues and stumbled upon several lzma errors for mdbox files of my own user. So I wrote a script which took the offset and size via "doveadm dump" und then dds the xz parts out of the broken mdboxes and extracts them via 7z. But I'm wondering why that happened at all. I stumpled upon [1] but there was no further discussion about it. Best Regards Bjoern [1]https://dovecot.org/pipermail/dovecot/2019-September/117007.html
