Hi, Sorry, I will update the issue on github, as the situation is more weird, than expected. I removed that message directly, reconstructed, squatter -> it dies with assertion failed, but the message earlier.
Maybe the next one is problematic? To be continue on github :) István 2022. 11. 28, hétfő keltezéssel 16.02-kor Pongrácz István ezt írta: > Hi, > > I just found a specific message, which cause the assertion failed > issue for me. > > It seems, this kind of issue cannot be solved using the reconstruct, > because technically it supposed to be valid email. > I know, because I sent that email some days ago from outside to test > something else. > > I reported here: > https://github.com/cyrusimap/cyrus-imapd/issues/4336 > > Cheers, > István > > > > > 2022. 11. 28, hétfő keltezéssel 14.58-kor Pongrácz István ezt írta: > > Hi, > > > > Just a quick update about the progress, regarding to this topic and > > give hint to others in similat situation. > > > > The standard squatter invocation still stopped, due to unknown > > issues, as described earlier. > > > > It seems I am able to fix this, using reconstruct for the > > problematic mailboxes, like this, as user cyrus: > > /usr/lib/cyrus/bin/reconstruct -r -x -u pongraczi > > > > Using the formula shown above, it seems I was able to fix 3 user's > > mailboxes already. > > The reported problems were, until now: > > * user.pongraczi.Archives.2013 uid 3197 not found (there were > > several emails - 8 - where uid not found) > > * some mails just missing (I assume I deleted them some weeks ago, but > > I forgot and I did not reconstruct yet) > > * one had wrong owner and permission (my bad, as root I moved around > > that file, so, the reason was me in one specific case) > > > > I used the following procedure to catch problematic mailboxes: > > * I used the strace to get logs about opened files (mails): > > strace -o squat2.log /usr/lib/cyrus/bin/squatter > > * when squattering stopped, due to problem, I was able to get the last > > message and maybe got a hint about the issue by grepping the log > > like this: > > cat squat2.log | grep "openat(AT_FDCWD" | tail > > the last line in my case: openat(AT_FDCWD, > > "/var/spool/cyrus/mail/uuid/j/9/j9x6ytfm3ain3r60yd9zcemw/469.", > > O_RDONLY) = 89 > > So, I got the file exactly the squattering died. > > Some cases I got ACCESS DENIED (was root:root instead of cyrus:mail) > > or no such file or directory messages > > * After reconstructing the user's mailbox using -r -x the squatter was > > able to squat these and move to the next mailbox instead of dying. > > > > Probably just reconstructing the full mail hierarchy using -r -x > > without any squatter testing could work and takes minimum time. > > > > If you need more information, please let me know. > > > > Cheers, > > István > > > > > > > > > > > > > > > > > > 2022. 11. 14, hétfő keltezéssel 07.39-kor Pongrácz István ezt írta: > > > Hi, > > > > > > Unfortunately not, but I cannot locate the specific emails, which > > > cause this. > > > I cannot provide too much useful information. > > > > > > It seems, when I put a lot of v in the options of squatter, it > > > can squatter all the messages, at least there is no assertion and > > > it did not stop. > > > > > > # incremental index changed mailboxes (fulltext) approximately every hour > > > squatter1 cmd="/usr/sbin/cyrus squatter -vvvvv -p -i" period=180 > > > > > > # reindex all mailboxes (fulltext) daily > > > squattera cmd="/usr/sbin/cyrus squatter -vvvvv -p" at=0130 > > > > > > In the other hand, the log contains additional info, around the > > > problematic folders. > > > > > > Like this: > > > 2022-11-13T01:32:11.937642+01:00 cyrus36 cyrus/squatter[19126]: indexing > > > mailbox user/csaba/Sent... > > > 2022-11-13T01:32:12.071267+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > opening document part '' > > > 2022-11-13T01:32:12.071595+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > writing 10 bytes into message 32091 > > > 2022-11-13T01:32:12.078024+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > opening document part 'h32091' > > > 2022-11-13T01:32:12.078185+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > writing 27 bytes into message 32091 > > > 2022-11-13T01:32:12.083676+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > opening document part 'f32092' > > > 2022-11-13T01:32:12.083837+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > writing 44 bytes into message 32092 > > > 2022-11-13T01:32:12.087780+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > opening document part 't32092' > > > 2022-11-13T01:32:12.088028+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > writing 48 bytes into message 32092 > > > 2022-11-13T01:32:12.090078+01:00 cyrus36 cyrus/squatter[19126]: squat: > > > opening document part 's32092' > > > > > > (red part is suspicious for me) > > > > > > But I do not find 1:1 between the additional logs and assertion > > > problems: seems more logs than assertion. (Maybe all of them > > > could cause assertion, but squatter normally dies at the first > > > one/account, so, the rest is invisible at this point.) > > > And unfortunately I was not able to find the exact mails, which > > > caused that. > > > Using mbexamine, I thought I found some lead, but the theory > > > failed when I tried to prove (emails with similar symptoms could > > > be squattered). > > > > > > I had no time to modify the squatter sourcecode with some more > > > debug (print the message id/filename) and compiling, yet. > > > I have the idea to print all filenames which opened for > > > squattering and where the assertion happened, I got the > > > problematic email. > > > It is still a theory. > > > > > > Do you think, I could any useful information at this moment? > > > > > > Thanks, > > > István > > > > > > > > > > > > > > > 2022. 11. 14, hétfő keltezéssel 10.43-kor ellie timoney ezt írta: > > > > Hi István, > > > > > > > > I haven't seen a bug report about this come through GitHub yet, > > > > did you end up solving the problem yourself? > > > > > > > > Cheers, > > > > > > > > ellie > > > > > > > > On Tue, 8 Nov 2022, at 11:51 PM, Pongrácz István wrote: > > > > > Thank you, I check it! > > > > > > > > > > cheers, > > > > > István > > > > > > > > > > > > > > > > > > > > > > > > > 2022. 11. 8, kedd keltezéssel 12.01-kor Nic Bernstein ezt > > > > > írta: > > > > > > I would suggest exploring a bit with 'mbexamine(8)' to see > > > > > > what it shows. See the man page here: > > > > > > > https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html > > > > > > Specifically, if you have either the UID or the sequence > > > > > > number of a message, you may use the "-u uid" or "-s > > > > > > seqnum" options to narrow down the output. > > > > > > > > > > > > If you then decide to create a bug report, the output from > > > > > > mbexamine(8) will be of use. > > > > > > > > > > > > Cheers, > > > > > > -nic > > > > > > On 11/8/22 11:55, Sebastian Hagedorn wrote: > > > > > > > > > > > > > > My suggestion would be to create a bug report on GitHub. > > > > > > > Cheers, > > > > > > > Sebastian > > > > > > > On 8 Nov 2022, at 12:47, Pongracz Istvan wrote: > > > > > > > > Hi, > > > > > > > > > > > > > > > > I tried to reconstruct one of the problematic account > > > > > > > > with this: > > > > > > > > > > > > > > > > /usr/lib/cyrus/bin/reconstruct -r -f -R -u pongraczi > > > > > > > > > > > > > > > > Didn't work, squatter still assert failured: > > > > > > > > > > > > > > > > cyrus/squatter[12044]: ERROR: message has more than > > > > > > > > 1000 header lines not caching any more > > > > > > > > fatal error: Internal error: assertion failed: > > > > > > > > imap/squat_internal.c: 134: v64 >= 0 > > > > > > > > > > > > > > > > In case of 2 out of 7 assertion issues, that more than > > > > > > > > 1000 header lines exists, but the rest (5) there is no > > > > > > > > other information, only the folder name available. > > > > > > > > > > > > > > > > Any idea? > > > > > > > > > > > > > > > > Thank you! > > > > > > > > IStván > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Pongrácz István <[email protected]> ezt írta > > > > > > > > (időpont: 2022. nov. 7., H, 16:13): > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > We use the cyrus version 3.6.0-beta3-1+b1 and I found > > > > > > > > > an issue with squatter. > > > > > > > > > In some accounts there are a folder, where the > > > > > > > > > squatter dies with fatal error and the whole squatter > > > > > > > > > process cannot move on. > > > > > > > > > > > > > > > > > > I mean, there are about 40 accounts, called a.... to > > > > > > > > > v.... > > > > > > > > > The squatter starts and can finish about 4 accounts, > > > > > > > > > up to user 'foo' where it run into a folder, I assume > > > > > > > > > it found an email and id dies with the following > > > > > > > > > message: > > > > > > > > > > > > > > > > > > process type:EVENT name:squatter1 > > > > > > > > > path:/usr/sbin/cyrus age:138.664s pid:31781 exited, > > > > > > > > > status 70 > > > > > > > > > > > > > > > > > > When I issue the command in command line, as cyrus > > > > > > > > > user > > > > > > > > > /usr/lib/cyrus/bin/squatter -v -p -u foo > > > > > > > > > > > > > > > > > > I got the following, more specific error message: > > > > > > > > > Indexing mailbox user/foo/Archives/2013... fatal > > > > > > > > > error: Internal error: assertion failed: > > > > > > > > > imap/squat_internal.c: 134: v64 >= 0 > > > > > > > > > > > > > > > > > > As I checked the squatter_internal.c it did not > > > > > > > > > change for years (github). > > > > > > > > > > > > > > > > > > The problem with this, the periodic squatter just > > > > > > > > > dies in the very beginning and other accounts never > > > > > > > > > will be squattered. > > > > > > > > > > > > > > > > > > This server populated using recent imapsync from old > > > > > > > > > cyrus server. > > > > > > > > > The used squatter is: squat (not xapian). > > > > > > > > > > > > > > > > > > Now I have to run squatter "manually", directly for > > > > > > > > > every user to get search function usable, more or > > > > > > > > > less. > > > > > > > > > (using the formula: /usr/lib/cyrus/bin/squatter -v -p > > > > > > > > > -u foo ) > > > > > > > > > > > > > > > > > > Do you know any kind of trick or method, how to > > > > > > > > > eliminate/solve this issue? > > > > > > > > > > > > > > > > > > Thank you! > > > > > > > > > István > > > > > > > > > > > > > > ------------------------------------------ Cyrus: Info Permalink: https://cyrus.topicbox.com/groups/info/Te0aac4ca5a14db4f-M323d08e92742679436f4d03d Delivery options: https://cyrus.topicbox.com/groups/info/subscription
